* [PATCH] fsmonitor: implement filesystem change listener for Linux
@ 2025-12-30 8:14 Paul Tarjan via GitGitGadget
2025-12-30 11:38 ` Junio C Hamano
2025-12-30 12:08 ` [PATCH v2] " Paul Tarjan via GitGitGadget
0 siblings, 2 replies; 129+ messages in thread
From: Paul Tarjan via GitGitGadget @ 2025-12-30 8:14 UTC (permalink / raw)
To: git; +Cc: Paul Tarjan, Paul Tarjan
From: Paul Tarjan <github@paulisageek.com>
Implement fsmonitor for Linux using the inotify API, bringing it to
feature parity with existing Windows and macOS implementations.
The Linux implementation uses inotify to monitor filesystem events.
Unlike macOS's FSEvents which can watch a single root directory,
inotify requires registering watches on every directory of interest.
The implementation carefully handles directory renames and moves
using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO
event pairs.
Key implementation details:
- Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring
- Maintains bidirectional hashmaps between watch descriptors and paths
for efficient event processing
- Handles directory creation, deletion, and renames dynamically
- Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs()
- Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote
- Creates batches lazily (only for actual file events, not cookies)
to avoid spurious sequence number increments
Build configuration:
- Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux
- Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset
- Adds HAVE_LINUX_MAGIC_H for filesystem type detection
Documentation updated to note that fsmonitor.socketDir is now supported
on both Mac OS and Linux, and adds a section about inotify watch limits.
Testing performed:
- Build succeeds with standard flags and SANITIZE=address
- All t7527-builtin-fsmonitor.sh tests pass on local filesystems
- Remote filesystem detection correctly rejects network mounts
Issues addressed from PR #1352 (git/git) review comments:
- GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from
scratch using statfs() and linux/magic.h constants (no GPLv3 code)
- Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup
- Unsafe hashmap iteration in dtor: Collect entries first, then modify
- Missing null checks in stop_async: Added proper guard conditions
- dirname() modifying argument: Create copy with xstrdup() first
- Non-portable f_fsid.__val: Use memcmp() for fsid comparison
- Missing worktree null check: Added BUG() for null worktree
- Header updates: Use git-compat-util.h, hash_to_hex_algop()
- Code style: Use xstrdup() not xmemdupz(), proper pointer style
Issues addressed from PR #1667 (git/git) review comments:
- EINTR handling: read() now handles both EAGAIN and EINTR
- Trailing pipe in log_mask_set: Added strbuf_strip_suffix()
- Unchecked add_watch return: Now logs failure in rename_dir()
- String building: Consolidated strbuf operations with strbuf_addf()
- Translation markers: Added _() to all error_errno() messages
Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta,
and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated
to work with the current codebase and address all review feedback.
Signed-off-by: Paul Tarjan <github@paulisageek.com>
---
fsmonitor: implement filesystem change listener for Linux
Implement fsmonitor for Linux using the inotify API, bringing it to
feature parity with existing Windows and macOS implementations.
The Linux implementation uses inotify to monitor filesystem events.
Unlike macOS's FSEvents which can watch a single root directory, inotify
requires registering watches on every directory of interest. The
implementation carefully handles directory renames and moves using
inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event
pairs.
Key implementation details:
* Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring
* Maintains bidirectional hashmaps between watch descriptors and paths
for efficient event processing
* Handles directory creation, deletion, and renames dynamically
* Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs()
* Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote
Build configuration:
* Enabled via FSMONITOR_DAEMON_BACKEND=linux and
FSMONITOR_OS_SETTINGS=linux
* Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset
* Adds HAVE_LINUX_MAGIC_H for filesystem type detection
Documentation updated to note that fsmonitor.socketDir is now supported
on both Mac OS and Linux, and adds a section about inotify watch limits.
Testing performed:
* Build succeeds with standard flags (NO_CURL, NO_GETTEXT, NO_TCLTK)
* Build succeeds with SANITIZE=address (AddressSanitizer)
* General tests pass (t0001-init, t3200-branch, etc.)
* fsmonitor tests require local filesystem (v9fs test environment is
correctly detected as remote and rejected)
* Manual testing confirms daemon starts and monitors filesystem when
configured with fsmonitor.allowremote=true and fsmonitor.socketdir
pointing to a local filesystem (tmpfs)
Issues addressed from PR #1352 (git/git) review comments:
* GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from
scratch using statfs() and linux/magic.h constants (no GPLv3 code)
* Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup
* Unsafe hashmap iteration in dtor: Collect entries first, then modify
* Missing null checks in stop_async: Added proper guard conditions
* dirname() modifying argument: Create copy with xstrdup() first
* Non-portable f_fsid.__val: Use memcmp() for fsid comparison
* Missing worktree null check: Added BUG() for null worktree
* Header updates: Use git-compat-util.h, hash_to_hex_algop()
* Code style: Use xstrdup() not xmemdupz(), proper pointer style
Issues addressed from PR #1667 (git/git) review comments:
* EINTR handling: read() now handles both EAGAIN and EINTR
* Trailing pipe in log_mask_set: Added strbuf_strip_suffix()
* Unchecked add_watch return: Now logs failure in rename_dir()
* String building: Consolidated strbuf operations with strbuf_addf()
* Translation markers: Added _() to all error_errno() messages
Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta,
and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to
work with the current codebase and address all review feedback.
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v1
Pull-Request: https://github.com/git/git/pull/2147
Documentation/config/fsmonitor--daemon.adoc | 4 +-
Documentation/git-fsmonitor--daemon.adoc | 28 +-
compat/fsmonitor/fsm-health-linux.c | 33 +
compat/fsmonitor/fsm-ipc-linux.c | 61 ++
compat/fsmonitor/fsm-listen-linux.c | 738 ++++++++++++++++++++
compat/fsmonitor/fsm-path-utils-linux.c | 224 ++++++
compat/fsmonitor/fsm-settings-linux.c | 71 ++
config.mak.uname | 10 +
contrib/buildsystems/CMakeLists.txt | 10 +
9 files changed, 1173 insertions(+), 6 deletions(-)
create mode 100644 compat/fsmonitor/fsm-health-linux.c
create mode 100644 compat/fsmonitor/fsm-ipc-linux.c
create mode 100644 compat/fsmonitor/fsm-listen-linux.c
create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c
create mode 100644 compat/fsmonitor/fsm-settings-linux.c
diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc
index 671f9b9462..6f8386e291 100644
--- a/Documentation/config/fsmonitor--daemon.adoc
+++ b/Documentation/config/fsmonitor--daemon.adoc
@@ -4,8 +4,8 @@ fsmonitor.allowRemote::
behavior. Only respected when `core.fsmonitor` is set to `true`.
fsmonitor.socketDir::
- This Mac OS-specific option, if set, specifies the directory in
+ This Mac OS and Linux-specific option, if set, specifies the directory in
which to create the Unix domain socket used for communication
between the fsmonitor daemon and various Git commands. The directory must
- reside on a native Mac OS filesystem. Only respected when `core.fsmonitor`
+ reside on a native filesystem. Only respected when `core.fsmonitor`
is set to `true`.
diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc
index 8fe5241b08..12fa866a64 100644
--- a/Documentation/git-fsmonitor--daemon.adoc
+++ b/Documentation/git-fsmonitor--daemon.adoc
@@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to
correctly with all network-mounted repositories, so such use is considered
experimental.
-On Mac OS, the inter-process communication (IPC) between various Git
+On Mac OS and Linux, the inter-process communication (IPC) between various Git
commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a
-special type of file -- which is supported by native Mac OS filesystems,
+special type of file -- which is supported by native Mac OS and Linux filesystems,
but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems
may or may not have the needed support; the fsmonitor daemon is not guaranteed
to work with these filesystems and such use is considered experimental.
@@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the
`.git` directory is on a network-mounted filesystem, it will instead be
created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a
network-mounted filesystem, in which case you must set the configuration
-variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native
+variable `fsmonitor.socketDir` to the path of a directory on a native
filesystem in which to create the socket file.
If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`)
-is on a native Mac OS file filesystem the fsmonitor daemon will report an
+is on a native filesystem the fsmonitor daemon will report an
error that will cause the daemon and the currently running command to exit.
+LINUX CAVEATS
+~~~~~~~~~~~~~
+
+On Linux, the fsmonitor daemon uses inotify to monitor filesystem events.
+The inotify system has per-user limits on the number of watches that can
+be created. The default limit is typically 8192 watches per user.
+
+For large repositories with many directories, you may need to increase
+this limit. Check the current limit with:
+
+ cat /proc/sys/fs/inotify/max_user_watches
+
+To temporarily increase the limit:
+
+ sudo sysctl fs.inotify.max_user_watches=65536
+
+To make the change permanent, add to `/etc/sysctl.conf`:
+
+ fs.inotify.max_user_watches=65536
+
CONFIGURATION
-------------
diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c
new file mode 100644
index 0000000000..43d67c4b8b
--- /dev/null
+++ b/compat/fsmonitor/fsm-health-linux.c
@@ -0,0 +1,33 @@
+#include "git-compat-util.h"
+#include "config.h"
+#include "fsmonitor-ll.h"
+#include "fsm-health.h"
+#include "fsmonitor--daemon.h"
+
+/*
+ * The Linux fsmonitor implementation uses inotify which has its own
+ * mechanisms for detecting filesystem unmount and other events that
+ * would require the daemon to shutdown. Therefore, we don't need
+ * a separate health thread like Windows does.
+ *
+ * These stub functions satisfy the interface requirements.
+ */
+
+int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED)
+{
+ return 0;
+}
+
+void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED)
+{
+ return;
+}
+
+void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED)
+{
+ return;
+}
+
+void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED)
+{
+}
diff --git a/compat/fsmonitor/fsm-ipc-linux.c b/compat/fsmonitor/fsm-ipc-linux.c
new file mode 100644
index 0000000000..d34a6419bc
--- /dev/null
+++ b/compat/fsmonitor/fsm-ipc-linux.c
@@ -0,0 +1,61 @@
+#define USE_THE_REPOSITORY_VARIABLE
+
+#include "git-compat-util.h"
+#include "config.h"
+#include "gettext.h"
+#include "hex.h"
+#include "path.h"
+#include "repository.h"
+#include "strbuf.h"
+#include "fsmonitor-ll.h"
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-path-utils.h"
+
+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc")
+
+const char *fsmonitor_ipc__get_path(struct repository *r)
+{
+ static const char *ipc_path = NULL;
+ git_SHA_CTX sha1ctx;
+ char *sock_dir = NULL;
+ struct strbuf ipc_file = STRBUF_INIT;
+ unsigned char hash[GIT_SHA1_RAWSZ];
+
+ if (!r)
+ BUG("No repository passed into fsmonitor_ipc__get_path");
+
+ if (ipc_path)
+ return ipc_path;
+
+ /* By default the socket file is created in the .git directory */
+ if (fsmonitor__is_fs_remote(r->gitdir) < 1) {
+ ipc_path = fsmonitor_ipc__get_default_path();
+ return ipc_path;
+ }
+
+ if (!r->worktree)
+ BUG("repository has no worktree");
+
+ git_SHA1_Init(&sha1ctx);
+ git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree));
+ git_SHA1_Final(hash, &sha1ctx);
+
+ repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir);
+
+ /* Create the socket file in either socketDir or $HOME */
+ if (sock_dir && *sock_dir) {
+ strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s",
+ sock_dir, hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1]));
+ } else {
+ strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s",
+ hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1]));
+ }
+ free(sock_dir);
+
+ ipc_path = interpolate_path(ipc_file.buf, 1);
+ if (!ipc_path)
+ die(_("Invalid path: %s"), ipc_file.buf);
+
+ strbuf_release(&ipc_file);
+ return ipc_path;
+}
diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c
new file mode 100644
index 0000000000..2593c834d3
--- /dev/null
+++ b/compat/fsmonitor/fsm-listen-linux.c
@@ -0,0 +1,738 @@
+#include "git-compat-util.h"
+#include "dir.h"
+#include "fsmonitor-ll.h"
+#include "fsm-listen.h"
+#include "fsmonitor--daemon.h"
+#include "fsmonitor-path-utils.h"
+#include "gettext.h"
+#include "simple-ipc.h"
+#include "string-list.h"
+#include "trace.h"
+
+#include <dirent.h>
+#include <fcntl.h>
+#include <poll.h>
+#include <sys/inotify.h>
+#include <sys/stat.h>
+
+/*
+ * Safe value to bitwise OR with rest of mask for
+ * kernels that do not support IN_MASK_CREATE
+ */
+#ifndef IN_MASK_CREATE
+#define IN_MASK_CREATE 0x00000000
+#endif
+
+enum shutdown_reason {
+ SHUTDOWN_CONTINUE = 0,
+ SHUTDOWN_STOP,
+ SHUTDOWN_ERROR,
+ SHUTDOWN_FORCE
+};
+
+struct watch_entry {
+ struct hashmap_entry ent;
+ int wd;
+ uint32_t cookie;
+ const char *dir;
+};
+
+struct rename_entry {
+ struct hashmap_entry ent;
+ time_t whence;
+ uint32_t cookie;
+ const char *dir;
+};
+
+struct fsm_listen_data {
+ int fd_inotify;
+ enum shutdown_reason shutdown;
+ struct hashmap watches;
+ struct hashmap renames;
+ struct hashmap revwatches;
+};
+
+static int watch_entry_cmp(const void *cmp_data UNUSED,
+ const struct hashmap_entry *eptr,
+ const struct hashmap_entry *entry_or_key,
+ const void *keydata UNUSED)
+{
+ const struct watch_entry *e1, *e2;
+
+ e1 = container_of(eptr, const struct watch_entry, ent);
+ e2 = container_of(entry_or_key, const struct watch_entry, ent);
+ return e1->wd != e2->wd;
+}
+
+static int revwatches_entry_cmp(const void *cmp_data UNUSED,
+ const struct hashmap_entry *eptr,
+ const struct hashmap_entry *entry_or_key,
+ const void *keydata UNUSED)
+{
+ const struct watch_entry *e1, *e2;
+
+ e1 = container_of(eptr, const struct watch_entry, ent);
+ e2 = container_of(entry_or_key, const struct watch_entry, ent);
+ return strcmp(e1->dir, e2->dir);
+}
+
+static int rename_entry_cmp(const void *cmp_data UNUSED,
+ const struct hashmap_entry *eptr,
+ const struct hashmap_entry *entry_or_key,
+ const void *keydata UNUSED)
+{
+ const struct rename_entry *e1, *e2;
+
+ e1 = container_of(eptr, const struct rename_entry, ent);
+ e2 = container_of(entry_or_key, const struct rename_entry, ent);
+ return e1->cookie != e2->cookie;
+}
+
+/*
+ * Register an inotify watch, add watch descriptor to path mapping
+ * and the reverse mapping.
+ */
+static int add_watch(const char *path, struct fsm_listen_data *data)
+{
+ const char *interned = strintern(path);
+ struct watch_entry *w1, *w2;
+
+ /* add the inotify watch, don't allow watches to be modified */
+ int wd = inotify_add_watch(data->fd_inotify, interned,
+ (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE)
+ ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN);
+ if (wd < 0) {
+ if (errno == ENOENT || errno == ENOTDIR)
+ return 0; /* directory was deleted or is not a directory */
+ return error_errno(_("inotify_add_watch('%s') failed"), interned);
+ }
+
+ /* add watch descriptor -> directory mapping */
+ CALLOC_ARRAY(w1, 1);
+ w1->wd = wd;
+ w1->dir = interned;
+ hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int)));
+ hashmap_add(&data->watches, &w1->ent);
+
+ /* add directory -> watch descriptor mapping */
+ CALLOC_ARRAY(w2, 1);
+ w2->wd = wd;
+ w2->dir = interned;
+ hashmap_entry_init(&w2->ent, strhash(w2->dir));
+ hashmap_add(&data->revwatches, &w2->ent);
+
+ return 0;
+}
+
+/*
+ * Remove the inotify watch, the watch descriptor to path mapping
+ * and the reverse mapping.
+ */
+static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data)
+{
+ struct watch_entry k1, k2, *w1, *w2;
+
+ /* remove watch, ignore error if kernel already did it */
+ if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL)
+ error_errno(_("inotify_rm_watch() failed"));
+
+ k1.wd = w->wd;
+ hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int)));
+ w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL);
+ if (!w1)
+ BUG("Double remove of watch for '%s'", w->dir);
+
+ if (w1->cookie)
+ BUG("Removing watch for '%s' which has a pending rename", w1->dir);
+
+ k2.dir = w->dir;
+ hashmap_entry_init(&k2.ent, strhash(k2.dir));
+ w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL);
+ if (!w2)
+ BUG("Double remove of reverse watch for '%s'", w->dir);
+
+ /* w1->dir and w2->dir are interned strings, we don't own them */
+ free(w1);
+ free(w2);
+}
+
+/*
+ * Check for stale directory renames.
+ *
+ * https://man7.org/linux/man-pages/man7/inotify.7.html
+ *
+ * Allow for some small timeout to account for the fact that insertion of the
+ * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that
+ * there may not be any IN_MOVED_TO event.
+ *
+ * If the IN_MOVED_TO event is not received within the timeout then events have
+ * been missed and the monitor is in an inconsistent state with respect to the
+ * filesystem.
+ */
+static int check_stale_dir_renames(struct hashmap *renames, time_t max_age)
+{
+ struct rename_entry *re;
+ struct hashmap_iter iter;
+
+ hashmap_for_each_entry(renames, &iter, re, ent) {
+ if (re->whence <= max_age)
+ return -1;
+ }
+ return 0;
+}
+
+/*
+ * Track pending renames.
+ *
+ * Tracking is done via an event cookie to watch descriptor mapping.
+ *
+ * A rename is not complete until matching an IN_MOVED_TO event is received
+ * for a corresponding IN_MOVED_FROM event.
+ */
+static void add_dir_rename(uint32_t cookie, const char *path,
+ struct fsm_listen_data *data)
+{
+ struct watch_entry k, *w;
+ struct rename_entry *re;
+
+ /* lookup the watch descriptor for the given path */
+ k.dir = path;
+ hashmap_entry_init(&k.ent, strhash(path));
+ w = hashmap_get_entry(&data->revwatches, &k, ent, NULL);
+ if (!w) {
+ /*
+ * This can happen in rare cases where the directory was
+ * moved before we had a chance to add a watch on it.
+ * Just ignore this rename.
+ */
+ trace_printf_key(&trace_fsmonitor,
+ "no watch found for rename from '%s'", path);
+ return;
+ }
+ w->cookie = cookie;
+
+ /* add the pending rename to match against later */
+ CALLOC_ARRAY(re, 1);
+ re->dir = w->dir;
+ re->cookie = w->cookie;
+ re->whence = time(NULL);
+ hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t)));
+ hashmap_add(&data->renames, &re->ent);
+}
+
+/*
+ * Handle directory renames
+ *
+ * Once an IN_MOVED_TO event is received, lookup the rename tracking information
+ * via the event cookie and use this information to update the watch.
+ */
+static void rename_dir(uint32_t cookie, const char *path,
+ struct fsm_listen_data *data)
+{
+ struct rename_entry rek, *re;
+ struct watch_entry k, *w;
+
+ /* lookup a pending rename to match */
+ rek.cookie = cookie;
+ hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t)));
+ re = hashmap_get_entry(&data->renames, &rek, ent, NULL);
+ if (re) {
+ k.dir = re->dir;
+ hashmap_entry_init(&k.ent, strhash(k.dir));
+ w = hashmap_get_entry(&data->revwatches, &k, ent, NULL);
+ if (w) {
+ w->cookie = 0; /* rename handled */
+ remove_watch(w, data);
+ if (add_watch(path, data))
+ trace_printf_key(&trace_fsmonitor,
+ "failed to add watch for renamed dir '%s'",
+ path);
+ } else {
+ /* Directory was moved out of watch tree */
+ trace_printf_key(&trace_fsmonitor,
+ "No matching watch for rename to '%s'", path);
+ }
+ hashmap_remove_entry(&data->renames, &rek, ent, NULL);
+ free(re);
+ } else {
+ /* Directory was moved from outside the watch tree */
+ trace_printf_key(&trace_fsmonitor,
+ "No matching cookie for rename to '%s'", path);
+ }
+}
+
+/*
+ * Recursively add watches to every directory under path
+ */
+static int register_inotify(const char *path,
+ struct fsmonitor_daemon_state *state,
+ struct fsmonitor_batch *batch)
+{
+ DIR *dir;
+ const char *rel;
+ struct strbuf current = STRBUF_INIT;
+ struct dirent *de;
+ struct stat fs;
+ int ret = -1;
+
+ dir = opendir(path);
+ if (!dir) {
+ if (errno == ENOENT || errno == ENOTDIR)
+ return 0; /* directory was deleted */
+ return error_errno(_("opendir('%s') failed"), path);
+ }
+
+ while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) {
+ strbuf_reset(¤t);
+ strbuf_addf(¤t, "%s/%s", path, de->d_name);
+ if (lstat(current.buf, &fs)) {
+ if (errno == ENOENT)
+ continue; /* file was deleted */
+ error_errno(_("lstat('%s') failed"), current.buf);
+ goto failed;
+ }
+
+ /* recurse into directory */
+ if (S_ISDIR(fs.st_mode)) {
+ if (add_watch(current.buf, state->listen_data))
+ goto failed;
+ if (register_inotify(current.buf, state, batch))
+ goto failed;
+ } else if (batch) {
+ rel = current.buf + state->path_worktree_watch.len + 1;
+ trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel);
+ fsmonitor_batch__add_path(batch, rel);
+ }
+ }
+ ret = 0;
+
+failed:
+ strbuf_release(¤t);
+ if (closedir(dir) < 0)
+ return error_errno(_("closedir('%s') failed"), path);
+ return ret;
+}
+
+static int em_rename_dir_from(uint32_t mask)
+{
+ return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM));
+}
+
+static int em_rename_dir_to(uint32_t mask)
+{
+ return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO));
+}
+
+static int em_remove_watch(uint32_t mask)
+{
+ return (mask & IN_DELETE_SELF);
+}
+
+static int em_dir_renamed(uint32_t mask)
+{
+ return ((mask & IN_ISDIR) && (mask & IN_MOVE));
+}
+
+static int em_dir_created(uint32_t mask)
+{
+ return ((mask & IN_ISDIR) && (mask & IN_CREATE));
+}
+
+static int em_dir_deleted(uint32_t mask)
+{
+ return ((mask & IN_ISDIR) && (mask & IN_DELETE));
+}
+
+static int em_force_shutdown(uint32_t mask)
+{
+ return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW);
+}
+
+static int em_ignore(uint32_t mask)
+{
+ return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF);
+}
+
+static void log_mask_set(const char *path, uint32_t mask)
+{
+ struct strbuf msg = STRBUF_INIT;
+
+ if (mask & IN_ACCESS)
+ strbuf_addstr(&msg, "IN_ACCESS|");
+ if (mask & IN_MODIFY)
+ strbuf_addstr(&msg, "IN_MODIFY|");
+ if (mask & IN_ATTRIB)
+ strbuf_addstr(&msg, "IN_ATTRIB|");
+ if (mask & IN_CLOSE_WRITE)
+ strbuf_addstr(&msg, "IN_CLOSE_WRITE|");
+ if (mask & IN_CLOSE_NOWRITE)
+ strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|");
+ if (mask & IN_OPEN)
+ strbuf_addstr(&msg, "IN_OPEN|");
+ if (mask & IN_MOVED_FROM)
+ strbuf_addstr(&msg, "IN_MOVED_FROM|");
+ if (mask & IN_MOVED_TO)
+ strbuf_addstr(&msg, "IN_MOVED_TO|");
+ if (mask & IN_CREATE)
+ strbuf_addstr(&msg, "IN_CREATE|");
+ if (mask & IN_DELETE)
+ strbuf_addstr(&msg, "IN_DELETE|");
+ if (mask & IN_DELETE_SELF)
+ strbuf_addstr(&msg, "IN_DELETE_SELF|");
+ if (mask & IN_MOVE_SELF)
+ strbuf_addstr(&msg, "IN_MOVE_SELF|");
+ if (mask & IN_UNMOUNT)
+ strbuf_addstr(&msg, "IN_UNMOUNT|");
+ if (mask & IN_Q_OVERFLOW)
+ strbuf_addstr(&msg, "IN_Q_OVERFLOW|");
+ if (mask & IN_IGNORED)
+ strbuf_addstr(&msg, "IN_IGNORED|");
+ if (mask & IN_ISDIR)
+ strbuf_addstr(&msg, "IN_ISDIR|");
+
+ strbuf_strip_suffix(&msg, "|");
+
+ trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s",
+ path, mask, msg.buf);
+
+ strbuf_release(&msg);
+}
+
+int fsm_listen__ctor(struct fsmonitor_daemon_state *state)
+{
+ int fd;
+ int ret = 0;
+ struct fsm_listen_data *data;
+
+ CALLOC_ARRAY(data, 1);
+ state->listen_data = data;
+ state->listen_error_code = -1;
+ data->shutdown = SHUTDOWN_ERROR;
+
+ fd = inotify_init1(O_NONBLOCK);
+ if (fd < 0) {
+ FREE_AND_NULL(state->listen_data);
+ return error_errno(_("inotify_init1() failed"));
+ }
+
+ data->fd_inotify = fd;
+
+ hashmap_init(&data->watches, watch_entry_cmp, NULL, 0);
+ hashmap_init(&data->renames, rename_entry_cmp, NULL, 0);
+ hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0);
+
+ if (add_watch(state->path_worktree_watch.buf, data))
+ ret = -1;
+ else if (register_inotify(state->path_worktree_watch.buf, state, NULL))
+ ret = -1;
+ else if (state->nr_paths_watching > 1) {
+ if (add_watch(state->path_gitdir_watch.buf, data))
+ ret = -1;
+ else if (register_inotify(state->path_gitdir_watch.buf, state, NULL))
+ ret = -1;
+ }
+
+ if (!ret) {
+ state->listen_error_code = 0;
+ data->shutdown = SHUTDOWN_CONTINUE;
+ }
+
+ return ret;
+}
+
+void fsm_listen__dtor(struct fsmonitor_daemon_state *state)
+{
+ struct fsm_listen_data *data;
+ struct hashmap_iter iter;
+ struct watch_entry *w;
+ struct watch_entry **to_remove;
+ size_t nr_to_remove = 0, alloc_to_remove = 0;
+ size_t i;
+ int fd;
+
+ if (!state || !state->listen_data)
+ return;
+
+ data = state->listen_data;
+ fd = data->fd_inotify;
+
+ /*
+ * Collect all entries first, then remove them.
+ * We can't modify the hashmap while iterating over it.
+ */
+ to_remove = NULL;
+ hashmap_for_each_entry(&data->watches, &iter, w, ent) {
+ ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove);
+ to_remove[nr_to_remove++] = w;
+ }
+
+ for (i = 0; i < nr_to_remove; i++) {
+ to_remove[i]->cookie = 0; /* ignore any pending renames */
+ remove_watch(to_remove[i], data);
+ }
+ free(to_remove);
+
+ hashmap_clear(&data->watches);
+
+ hashmap_clear(&data->revwatches); /* remove_watch freed the entries */
+
+ hashmap_clear_and_free(&data->renames, struct rename_entry, ent);
+
+ FREE_AND_NULL(state->listen_data);
+
+ if (fd && (close(fd) < 0))
+ error_errno(_("closing inotify file descriptor failed"));
+}
+
+void fsm_listen__stop_async(struct fsmonitor_daemon_state *state)
+{
+ if (state && state->listen_data &&
+ state->listen_data->shutdown == SHUTDOWN_CONTINUE)
+ state->listen_data->shutdown = SHUTDOWN_STOP;
+}
+
+/*
+ * Process a single inotify event and queue for publication.
+ */
+static int process_event(const char *path,
+ const struct inotify_event *event,
+ struct fsmonitor_batch **batch,
+ struct string_list *cookie_list,
+ struct fsmonitor_daemon_state *state)
+{
+ const char *rel;
+ const char *last_sep;
+
+ switch (fsmonitor_classify_path_absolute(state, path)) {
+ case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX:
+ case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX:
+ /* Use just the filename of the cookie file. */
+ last_sep = find_last_dir_sep(path);
+ string_list_append(cookie_list,
+ last_sep ? last_sep + 1 : path);
+ break;
+ case IS_INSIDE_DOT_GIT:
+ case IS_INSIDE_GITDIR:
+ break;
+ case IS_DOT_GIT:
+ case IS_GITDIR:
+ /*
+ * If .git directory is deleted or renamed away,
+ * we have to quit.
+ */
+ if (em_dir_deleted(event->mask)) {
+ trace_printf_key(&trace_fsmonitor,
+ "event: gitdir removed");
+ state->listen_data->shutdown = SHUTDOWN_FORCE;
+ goto done;
+ }
+
+ if (em_dir_renamed(event->mask)) {
+ trace_printf_key(&trace_fsmonitor,
+ "event: gitdir renamed");
+ state->listen_data->shutdown = SHUTDOWN_FORCE;
+ goto done;
+ }
+ break;
+ case IS_WORKDIR_PATH:
+ /* normal events in the working directory */
+ if (trace_pass_fl(&trace_fsmonitor))
+ log_mask_set(path, event->mask);
+
+ if (!*batch)
+ *batch = fsmonitor_batch__new();
+
+ rel = path + state->path_worktree_watch.len + 1;
+ fsmonitor_batch__add_path(*batch, rel);
+
+ if (em_dir_deleted(event->mask))
+ break;
+
+ /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */
+ if (em_rename_dir_from(event->mask))
+ add_dir_rename(event->cookie, path, state->listen_data);
+
+ /* received IN_MOVE_TO, update watch to reflect new path */
+ if (em_rename_dir_to(event->mask)) {
+ rename_dir(event->cookie, path, state->listen_data);
+ if (register_inotify(path, state, *batch)) {
+ state->listen_data->shutdown = SHUTDOWN_ERROR;
+ goto done;
+ }
+ }
+
+ if (em_dir_created(event->mask)) {
+ if (add_watch(path, state->listen_data)) {
+ state->listen_data->shutdown = SHUTDOWN_ERROR;
+ goto done;
+ }
+ if (register_inotify(path, state, *batch)) {
+ state->listen_data->shutdown = SHUTDOWN_ERROR;
+ goto done;
+ }
+ }
+ break;
+ case IS_OUTSIDE_CONE:
+ default:
+ trace_printf_key(&trace_fsmonitor,
+ "ignoring '%s'", path);
+ break;
+ }
+ return 0;
+done:
+ return -1;
+}
+
+/*
+ * Read the inotify event stream and pre-process events before further
+ * processing and eventual publishing.
+ */
+static void handle_events(struct fsmonitor_daemon_state *state)
+{
+ /* See https://man7.org/linux/man-pages/man7/inotify.7.html */
+ char buf[4096]
+ __attribute__ ((aligned(__alignof__(struct inotify_event))));
+
+ struct hashmap *watches = &state->listen_data->watches;
+ struct fsmonitor_batch *batch = NULL;
+ struct string_list cookie_list = STRING_LIST_INIT_DUP;
+ struct watch_entry k, *w;
+ struct strbuf path = STRBUF_INIT;
+ const struct inotify_event *event;
+ int fd = state->listen_data->fd_inotify;
+ ssize_t len;
+ char *ptr, *p;
+
+ for (;;) {
+ len = read(fd, buf, sizeof(buf));
+ if (len == -1) {
+ if (errno == EAGAIN || errno == EINTR)
+ goto done;
+ error_errno(_("reading inotify message stream failed"));
+ state->listen_data->shutdown = SHUTDOWN_ERROR;
+ goto done;
+ }
+
+ /* nothing to read */
+ if (len == 0)
+ goto done;
+
+ /* Loop over all events in the buffer. */
+ for (ptr = buf; ptr < buf + len;
+ ptr += sizeof(struct inotify_event) + event->len) {
+
+ event = (const struct inotify_event *)ptr;
+
+ if (em_ignore(event->mask))
+ continue;
+
+ /* File system was unmounted or event queue overflowed */
+ if (em_force_shutdown(event->mask)) {
+ if (trace_pass_fl(&trace_fsmonitor))
+ log_mask_set("Forcing shutdown", event->mask);
+ state->listen_data->shutdown = SHUTDOWN_FORCE;
+ goto done;
+ }
+
+ k.wd = event->wd;
+ hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int)));
+
+ w = hashmap_get_entry(watches, &k, ent, NULL);
+ if (!w) {
+ /* Watch was removed, skip event */
+ continue;
+ }
+
+ /* directory watch was removed */
+ if (em_remove_watch(event->mask)) {
+ remove_watch(w, state->listen_data);
+ continue;
+ }
+
+ strbuf_reset(&path);
+ strbuf_addf(&path, "%s/%s", w->dir, event->name);
+
+ p = fsmonitor__resolve_alias(path.buf, &state->alias);
+ if (!p)
+ p = strbuf_detach(&path, NULL);
+
+ if (process_event(p, event, &batch, &cookie_list, state)) {
+ free(p);
+ goto done;
+ }
+ free(p);
+ }
+ strbuf_reset(&path);
+ fsmonitor_publish(state, batch, &cookie_list);
+ string_list_clear(&cookie_list, 0);
+ batch = NULL;
+ }
+done:
+ strbuf_release(&path);
+ fsmonitor_batch__free_list(batch);
+ string_list_clear(&cookie_list, 0);
+}
+
+/*
+ * Non-blocking read of the inotify events stream. The inotify fd is polled
+ * frequently to help minimize the number of queue overflows.
+ */
+void fsm_listen__loop(struct fsmonitor_daemon_state *state)
+{
+ int poll_num;
+ const int interval = 1000;
+ time_t checked = time(NULL);
+ struct pollfd fds[1];
+
+ fds[0].fd = state->listen_data->fd_inotify;
+ fds[0].events = POLLIN;
+
+ /*
+ * Our fs event listener is now running, so it's safe to start
+ * serving client requests.
+ */
+ ipc_server_start_async(state->ipc_server_data);
+
+ for (;;) {
+ switch (state->listen_data->shutdown) {
+ case SHUTDOWN_CONTINUE:
+ poll_num = poll(fds, 1, 1);
+ if (poll_num == -1) {
+ if (errno == EINTR)
+ continue;
+ error_errno(_("polling inotify message stream failed"));
+ state->listen_data->shutdown = SHUTDOWN_ERROR;
+ continue;
+ }
+
+ if ((time(NULL) - checked) >= interval) {
+ checked = time(NULL);
+ if (check_stale_dir_renames(&state->listen_data->renames,
+ checked - interval)) {
+ trace_printf_key(&trace_fsmonitor,
+ "Missed IN_MOVED_TO events, forcing shutdown");
+ state->listen_data->shutdown = SHUTDOWN_FORCE;
+ continue;
+ }
+ }
+
+ if (poll_num > 0 && (fds[0].revents & POLLIN))
+ handle_events(state);
+
+ continue;
+ case SHUTDOWN_ERROR:
+ state->listen_error_code = -1;
+ ipc_server_stop_async(state->ipc_server_data);
+ break;
+ case SHUTDOWN_FORCE:
+ state->listen_error_code = 0;
+ ipc_server_stop_async(state->ipc_server_data);
+ break;
+ case SHUTDOWN_STOP:
+ default:
+ state->listen_error_code = 0;
+ break;
+ }
+ return;
+ }
+}
diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c
new file mode 100644
index 0000000000..d0bc98b0a9
--- /dev/null
+++ b/compat/fsmonitor/fsm-path-utils-linux.c
@@ -0,0 +1,224 @@
+#include "git-compat-util.h"
+#include "fsmonitor-ll.h"
+#include "fsmonitor-path-utils.h"
+#include "gettext.h"
+#include "trace.h"
+
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/statfs.h>
+
+#ifdef HAVE_LINUX_MAGIC_H
+#include <linux/magic.h>
+#endif
+
+/*
+ * Filesystem magic numbers for remote filesystems.
+ * Defined here if not available in linux/magic.h.
+ */
+#ifndef CIFS_SUPER_MAGIC
+#define CIFS_SUPER_MAGIC 0xff534d42
+#endif
+#ifndef SMB_SUPER_MAGIC
+#define SMB_SUPER_MAGIC 0x517b
+#endif
+#ifndef SMB2_SUPER_MAGIC
+#define SMB2_SUPER_MAGIC 0xfe534d42
+#endif
+#ifndef NFS_SUPER_MAGIC
+#define NFS_SUPER_MAGIC 0x6969
+#endif
+#ifndef AFS_SUPER_MAGIC
+#define AFS_SUPER_MAGIC 0x5346414f
+#endif
+#ifndef CODA_SUPER_MAGIC
+#define CODA_SUPER_MAGIC 0x73757245
+#endif
+#ifndef V9FS_MAGIC
+#define V9FS_MAGIC 0x01021997
+#endif
+#ifndef FUSE_SUPER_MAGIC
+#define FUSE_SUPER_MAGIC 0x65735546
+#endif
+
+/*
+ * Check if filesystem type is a remote filesystem.
+ */
+static int is_remote_fs(unsigned long f_type)
+{
+ switch (f_type) {
+ case CIFS_SUPER_MAGIC:
+ case SMB_SUPER_MAGIC:
+ case SMB2_SUPER_MAGIC:
+ case NFS_SUPER_MAGIC:
+ case AFS_SUPER_MAGIC:
+ case CODA_SUPER_MAGIC:
+ case V9FS_MAGIC:
+ case FUSE_SUPER_MAGIC:
+ return 1;
+ default:
+ return 0;
+ }
+}
+
+/*
+ * Get the filesystem type name for logging purposes.
+ */
+static const char *get_fs_typename(unsigned long f_type)
+{
+ switch (f_type) {
+ case CIFS_SUPER_MAGIC:
+ return "cifs";
+ case SMB_SUPER_MAGIC:
+ return "smb";
+ case SMB2_SUPER_MAGIC:
+ return "smb2";
+ case NFS_SUPER_MAGIC:
+ return "nfs";
+ case AFS_SUPER_MAGIC:
+ return "afs";
+ case CODA_SUPER_MAGIC:
+ return "coda";
+ case V9FS_MAGIC:
+ return "9p";
+ case FUSE_SUPER_MAGIC:
+ return "fuse";
+ default:
+ return "unknown";
+ }
+}
+
+/*
+ * Find the mount point for a given path by reading /proc/mounts.
+ * Returns the filesystem type for the longest matching mount point.
+ */
+static char *find_mount(const char *path, struct statfs *fs)
+{
+ FILE *fp;
+ struct strbuf line = STRBUF_INIT;
+ struct strbuf match = STRBUF_INIT;
+ struct strbuf fstype = STRBUF_INIT;
+ char *result = NULL;
+ struct statfs path_fs;
+
+ if (statfs(path, &path_fs) < 0)
+ return NULL;
+
+ fp = fopen("/proc/mounts", "r");
+ if (!fp)
+ return NULL;
+
+ while (strbuf_getline(&line, fp) != EOF) {
+ char *fields[6];
+ char *p = line.buf;
+ int i;
+
+ /* Parse mount entry: device mountpoint fstype options dump pass */
+ for (i = 0; i < 6 && p; i++) {
+ fields[i] = p;
+ p = strchr(p, ' ');
+ if (p)
+ *p++ = '\0';
+ }
+
+ if (i >= 3) {
+ const char *mountpoint = fields[1];
+ const char *type = fields[2];
+ struct statfs mount_fs;
+
+ /* Check if this mount point is a prefix of our path */
+ if (starts_with(path, mountpoint) &&
+ (path[strlen(mountpoint)] == '/' ||
+ path[strlen(mountpoint)] == '\0')) {
+ /* Check if filesystem ID matches */
+ if (statfs(mountpoint, &mount_fs) == 0 &&
+ !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid,
+ sizeof(mount_fs.f_fsid))) {
+ /* Keep the longest matching mount point */
+ if (strlen(mountpoint) > match.len) {
+ strbuf_reset(&match);
+ strbuf_addstr(&match, mountpoint);
+ strbuf_reset(&fstype);
+ strbuf_addstr(&fstype, type);
+ *fs = mount_fs;
+ }
+ }
+ }
+ }
+ }
+
+ fclose(fp);
+ strbuf_release(&line);
+ strbuf_release(&match);
+
+ if (fstype.len)
+ result = strbuf_detach(&fstype, NULL);
+ else
+ strbuf_release(&fstype);
+
+ return result;
+}
+
+int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info)
+{
+ struct statfs fs;
+
+ if (statfs(path, &fs) == -1) {
+ int saved_errno = errno;
+ trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s",
+ path, strerror(saved_errno));
+ errno = saved_errno;
+ return -1;
+ }
+
+ trace_printf_key(&trace_fsmonitor,
+ "statfs('%s') [type 0x%08lx]",
+ path, (unsigned long)fs.f_type);
+
+ fs_info->is_remote = is_remote_fs(fs.f_type);
+
+ /*
+ * Try to get filesystem type from /proc/mounts for a more
+ * descriptive name.
+ */
+ fs_info->typename = find_mount(path, &fs);
+ if (!fs_info->typename)
+ fs_info->typename = xstrdup(get_fs_typename(fs.f_type));
+
+ trace_printf_key(&trace_fsmonitor,
+ "'%s' is_remote: %d, typename: %s",
+ path, fs_info->is_remote, fs_info->typename);
+
+ return 0;
+}
+
+int fsmonitor__is_fs_remote(const char *path)
+{
+ struct fs_info fs;
+
+ if (fsmonitor__get_fs_info(path, &fs))
+ return -1;
+
+ free(fs.typename);
+
+ return fs.is_remote;
+}
+
+/*
+ * No-op for Linux - we don't have firmlinks like macOS.
+ */
+int fsmonitor__get_alias(const char *path UNUSED,
+ struct alias_info *info UNUSED)
+{
+ return 0;
+}
+
+/*
+ * No-op for Linux - we don't have firmlinks like macOS.
+ */
+char *fsmonitor__resolve_alias(const char *path UNUSED,
+ const struct alias_info *info UNUSED)
+{
+ return NULL;
+}
diff --git a/compat/fsmonitor/fsm-settings-linux.c b/compat/fsmonitor/fsm-settings-linux.c
new file mode 100644
index 0000000000..23e7442d0c
--- /dev/null
+++ b/compat/fsmonitor/fsm-settings-linux.c
@@ -0,0 +1,71 @@
+#include "git-compat-util.h"
+#include "config.h"
+#include "fsmonitor-ll.h"
+#include "fsmonitor-ipc.h"
+#include "fsmonitor-settings.h"
+#include "fsmonitor-path-utils.h"
+
+#include <libgen.h>
+
+/*
+ * For the builtin FSMonitor, we create the Unix domain socket for the
+ * IPC in the .git directory. If the working directory is remote,
+ * then the socket will be created on the remote file system. This
+ * can fail if the remote file system does not support UDS file types
+ * (e.g. smbfs to a Windows server) or if the remote kernel does not
+ * allow a non-local process to bind() the socket. (These problems
+ * could be fixed by moving the UDS out of the .git directory and to a
+ * well-known local directory on the client machine, but care should
+ * be taken to ensure that $HOME is actually local and not a managed
+ * file share.)
+ *
+ * FAT32 and NTFS working directories are problematic too.
+ *
+ * The builtin FSMonitor uses a Unix domain socket in the .git
+ * directory for IPC. These Windows drive formats do not support
+ * Unix domain sockets, so mark them as incompatible for the daemon.
+ */
+static enum fsmonitor_reason check_uds_volume(struct repository *r)
+{
+ struct fs_info fs;
+ const char *ipc_path = fsmonitor_ipc__get_path(r);
+ char *path;
+ char *dir;
+
+ /*
+ * Create a copy for dirname() since it may modify its argument.
+ */
+ path = xstrdup(ipc_path);
+ dir = dirname(path);
+
+ if (fsmonitor__get_fs_info(dir, &fs) == -1) {
+ free(path);
+ return FSMONITOR_REASON_ERROR;
+ }
+
+ free(path);
+
+ if (fs.is_remote ||
+ !strcmp(fs.typename, "msdos") ||
+ !strcmp(fs.typename, "ntfs") ||
+ !strcmp(fs.typename, "vfat")) {
+ free(fs.typename);
+ return FSMONITOR_REASON_NOSOCKETS;
+ }
+
+ free(fs.typename);
+ return FSMONITOR_REASON_OK;
+}
+
+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc)
+{
+ enum fsmonitor_reason reason;
+
+ if (ipc) {
+ reason = check_uds_volume(r);
+ if (reason != FSMONITOR_REASON_OK)
+ return reason;
+ }
+
+ return FSMONITOR_REASON_OK;
+}
diff --git a/config.mak.uname b/config.mak.uname
index 1691c6ae6e..a0fd0752a3 100644
--- a/config.mak.uname
+++ b/config.mak.uname
@@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux)
BASIC_CFLAGS += -std=c99
endif
LINK_FUZZ_PROGRAMS = YesPlease
+
+ # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require
+ # Unix domain sockets and PThreads.
+ ifndef NO_PTHREADS
+ ifndef NO_UNIX_SOCKETS
+ FSMONITOR_DAEMON_BACKEND = linux
+ FSMONITOR_OS_SETTINGS = linux
+ BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H
+ endif
+ endif
endif
ifeq ($(uname_S),GNU/kFreeBSD)
HAVE_ALLOCA_H = YesPlease
diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt
index 28877feb9d..141b05acb0 100644
--- a/contrib/buildsystems/CMakeLists.txt
+++ b/contrib/buildsystems/CMakeLists.txt
@@ -308,6 +308,16 @@ if(SUPPORTS_SIMPLE_IPC)
add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c)
+ elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux")
+ add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND)
+ add_compile_definitions(HAVE_LINUX_MAGIC_H)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-linux.c)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c)
+
+ add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS)
+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-linux.c)
endif()
endif()
base-commit: 7c7698a654a7a0031f65b0ab0c1c4e438e95df60
--
gitgitgadget
^ permalink raw reply related [flat|nested] 129+ messages in thread* Re: [PATCH] fsmonitor: implement filesystem change listener for Linux 2025-12-30 8:14 [PATCH] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2025-12-30 11:38 ` Junio C Hamano 2025-12-30 12:08 ` [PATCH v2] " Paul Tarjan via GitGitGadget 1 sibling, 0 replies; 129+ messages in thread From: Junio C Hamano @ 2025-12-30 11:38 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan Exciting. It seems to die with leaks when "make SANITIZE=leak test" is run, though. Initialized empty Git repository in /home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit/.git/ fsmonitor-daemon is not watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit' builtin:0.145521.20251230T113644.793433Z:0Q/Q {"event":"data","sid":"20251230T113644.762195Z-H3cfff1b1-P0002386f","thread":"main","time":"2025-12-30T11:36:44.813131Z","file":"fsmonitor-ipc.c","line":99,"t_abs":0.052581,"t_rel":0.048830,"nesting":2,"category":"fsm_client","key":"query/response-length","value":"45"} fsmonitor-daemon is watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit' fsmonitor-daemon is not watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor/test_implicit' fatal: fsmonitor--daemon is not running not ok 2 - implicit daemon start # # test_when_finished "stop_daemon_delete_repo test_implicit" && # # git init test_implicit && # test_must_fail git -C test_implicit fsmonitor--daemon status && # # # query will implicitly start the daemon. # # # # for test-script simplicity, we send a V1 timestamp rather than # # a V2 token. either way, the daemon response to any query contains # # a new V2 token. (the daemon may complain that we sent a V1 request, # # but this test case is only concerned with whether the daemon was # # implicitly started.) # # GIT_TRACE2_EVENT="$PWD/.git/trace" \ # test-tool -C test_implicit fsmonitor-client query --token 0 >actual && # nul_to_q <actual >actual.filtered && # grep "builtin:" actual.filtered && # # # confirm that a daemon was started in the background. # # # # since the mechanism for starting the background daemon is platform # # dependent, just confirm that the foreground command received a # # response from the daemon. # # have_t2_data_event fsm_client query/response-length <.git/trace && # # git -C test_implicit fsmonitor--daemon status && # git -C test_implicit fsmonitor--daemon stop && # test_must_fail git -C test_implicit fsmonitor--daemon status # 1..2 ================================================================= ==git==145489==ERROR: LeakSanitizer: detected memory leaks Direct leak of 512 byte(s) in 1 object(s) allocated from: #0 0x56179e6ce042 in calloc (git+0x8c042) (BuildId: de5ce3c9d0b0c09380c910e6a9eb181e324abde6) #1 0x56179ea0aef4 in xcalloc wrapper.c:154:8 #2 0x56179e8b17b7 in alloc_table hashmap.c:79:2 #3 0x56179e8b174c in hashmap_init hashmap.c:168:2 #4 0x56179e73e6fe in fsmonitor_run_daemon builtin/fsmonitor--daemon.c:1288:2 #5 0x56179e73e141 in try_to_run_foreground_daemon builtin/fsmonitor--daemon.c:1448:11 #6 0x56179e73dc44 in cmd_fsmonitor__daemon builtin/fsmonitor--daemon.c:1584:12 #7 0x56179e6d2c8a in run_builtin git.c:506:11 #8 0x56179e6d1910 in handle_builtin git.c:779:9 #9 0x56179e6d2747 in run_argv git.c:862:4 #10 0x56179e6d169b in cmd_main git.c:984:19 #11 0x56179e7f7a7a in main common-main.c:9:11 #12 0x7f091ea66ca7 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 #13 0x7f091ea66d64 in __libc_start_main csu/../csu/libc-start.c:360:3 #14 0x56179e69e280 in _start (git+0x5c280) (BuildId: de5ce3c9d0b0c09380c910e6a9eb181e324abde6) ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v2] fsmonitor: implement filesystem change listener for Linux 2025-12-30 8:14 [PATCH] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2025-12-30 11:38 ` Junio C Hamano @ 2025-12-30 12:08 ` Paul Tarjan via GitGitGadget 2025-12-30 12:55 ` [PATCH v3] " Paul Tarjan via GitGitGadget 2025-12-30 15:37 ` [PATCH v2] " Junio C Hamano 1 sibling, 2 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2025-12-30 12:08 UTC (permalink / raw) To: git; +Cc: Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring - Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing - Handles directory creation, deletion, and renames dynamically - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote - Creates batches lazily (only for actual file events, not cookies) to avoid spurious sequence number increments Build configuration: - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset - Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Testing performed: - Build succeeds with standard flags and SANITIZE=address - All t7527-builtin-fsmonitor.sh tests pass on local filesystems - Remote filesystem detection correctly rejects network mounts Issues addressed from PR #1352 (git/git) review comments: - GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) - Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup - Unsafe hashmap iteration in dtor: Collect entries first, then modify - Missing null checks in stop_async: Added proper guard conditions - dirname() modifying argument: Create copy with xstrdup() first - Non-portable f_fsid.__val: Use memcmp() for fsid comparison - Missing worktree null check: Added BUG() for null worktree - Header updates: Use git-compat-util.h, hash_to_hex_algop() - Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: - EINTR handling: read() now handles both EAGAIN and EINTR - Trailing pipe in log_mask_set: Added strbuf_strip_suffix() - Unchecked add_watch return: Now logs failure in rename_dir() - String building: Consolidated strbuf operations with strbuf_addf() - Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- fsmonitor: implement filesystem change listener for Linux Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: * Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring * Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing * Handles directory creation, deletion, and renames dynamically * Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() * Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote Build configuration: * Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux * Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset * Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Testing performed: * Build succeeds with standard flags (NO_CURL, NO_GETTEXT, NO_TCLTK) * Build succeeds with SANITIZE=address (AddressSanitizer) * General tests pass (t0001-init, t3200-branch, etc.) * fsmonitor tests require local filesystem (v9fs test environment is correctly detected as remote and rejected) * Manual testing confirms daemon starts and monitors filesystem when configured with fsmonitor.allowremote=true and fsmonitor.socketdir pointing to a local filesystem (tmpfs) Issues addressed from PR #1352 (git/git) review comments: * GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) * Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup * Unsafe hashmap iteration in dtor: Collect entries first, then modify * Missing null checks in stop_async: Added proper guard conditions * dirname() modifying argument: Create copy with xstrdup() first * Non-portable f_fsid.__val: Use memcmp() for fsid comparison * Missing worktree null check: Added BUG() for null worktree * Header updates: Use git-compat-util.h, hash_to_hex_algop() * Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: * EINTR handling: read() now handles both EAGAIN and EINTR * Trailing pipe in log_mask_set: Added strbuf_strip_suffix() * Unchecked add_watch return: Now logs failure in rename_dir() * String building: Consolidated strbuf operations with strbuf_addf() * Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v1: * Fixed memory leak in builtin/fsmonitor--daemon.c: Added hashmap_clear(&state.cookies) in the cleanup section of fsmonitor_run_daemon() to free the cookies hashmap that was initialized but never released. This fixes the 512-byte leak detected by LeakSanitizer. * Removed 9p (Plan 9) from remote filesystem list in compat/fsmonitor/fsm-path-utils-linux.c: Removed V9FS_MAGIC from is_remote_fs() because inotify works correctly on 9p filesystems. This was incorrectly preventing fsmonitor from running in 9p-based container environments where inotify is fully functional. Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v2 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v1: 1: aa606458a6 ! 1: 7a7fe25a06 fsmonitor: implement filesystem change listener for Linux @@ Documentation/git-fsmonitor--daemon.adoc: By default, the socket is created in t ------------- + ## builtin/fsmonitor--daemon.c ## +@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void) + done: + pthread_cond_destroy(&state.cookies_cond); + pthread_mutex_destroy(&state.main_lock); ++ hashmap_clear(&state.cookies); + fsm_listen__dtor(&state); + fsm_health__dtor(&state); + + ## compat/fsmonitor/fsm-health-linux.c (new) ## @@ +#include "git-compat-util.h" @@ compat/fsmonitor/fsm-path-utils-linux.c (new) + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: -+ case V9FS_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- builtin/fsmonitor--daemon.c | 1 + compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-ipc-linux.c | 61 ++ compat/fsmonitor/fsm-listen-linux.c | 738 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 223 ++++++ compat/fsmonitor/fsm-settings-linux.c | 71 ++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 10 + 10 files changed, 1173 insertions(+), 6 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-ipc-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c create mode 100644 compat/fsmonitor/fsm-settings-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..92c49f6ab2 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1405,6 +1405,7 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + hashmap_clear(&state.cookies); fsm_listen__dtor(&state); fsm_health__dtor(&state); diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-ipc-linux.c b/compat/fsmonitor/fsm-ipc-linux.c new file mode 100644 index 0000000000..d34a6419bc --- /dev/null +++ b/compat/fsmonitor/fsm-ipc-linux.c @@ -0,0 +1,61 @@ +#define USE_THE_REPOSITORY_VARIABLE + +#include "git-compat-util.h" +#include "config.h" +#include "gettext.h" +#include "hex.h" +#include "path.h" +#include "repository.h" +#include "strbuf.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" + +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path(struct repository *r) +{ + static const char *ipc_path = NULL; + git_SHA_CTX sha1ctx; + char *sock_dir = NULL; + struct strbuf ipc_file = STRBUF_INIT; + unsigned char hash[GIT_SHA1_RAWSZ]; + + if (!r) + BUG("No repository passed into fsmonitor_ipc__get_path"); + + if (ipc_path) + return ipc_path; + + /* By default the socket file is created in the .git directory */ + if (fsmonitor__is_fs_remote(r->gitdir) < 1) { + ipc_path = fsmonitor_ipc__get_default_path(); + return ipc_path; + } + + if (!r->worktree) + BUG("repository has no worktree"); + + git_SHA1_Init(&sha1ctx); + git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); + git_SHA1_Final(hash, &sha1ctx); + + repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir); + + /* Create the socket file in either socketDir or $HOME */ + if (sock_dir && *sock_dir) { + strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s", + sock_dir, hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } else { + strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", + hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } + free(sock_dir); + + ipc_path = interpolate_path(ipc_file.buf, 1); + if (!ipc_path) + die(_("Invalid path: %s"), ipc_file.buf); + + strbuf_release(&ipc_file); + return ipc_path; +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..2593c834d3 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,738 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <dirent.h> +#include <fcntl.h> +#include <poll.h> +#include <sys/inotify.h> +#include <sys/stat.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("Double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("Removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("Double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("Forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + const int interval = 1000; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 1); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "Missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..fc4acbc20d --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,223 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <errno.h> +#include <stdio.h> +#include <string.h> +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef V9FS_MAGIC +#define V9FS_MAGIC 0x01021997 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Get the filesystem type name for logging purposes. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case V9FS_MAGIC: + return "9p"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * Returns the filesystem type for the longest matching mount point. + */ +static char *find_mount(const char *path, struct statfs *fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + struct statfs path_fs; + + if (statfs(path, &path_fs) < 0) + return NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + *fs = mount_fs; + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/compat/fsmonitor/fsm-settings-linux.c b/compat/fsmonitor/fsm-settings-linux.c new file mode 100644 index 0000000000..23e7442d0c --- /dev/null +++ b/compat/fsmonitor/fsm-settings-linux.c @@ -0,0 +1,71 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-settings.h" +#include "fsmonitor-path-utils.h" + +#include <libgen.h> + +/* + * For the builtin FSMonitor, we create the Unix domain socket for the + * IPC in the .git directory. If the working directory is remote, + * then the socket will be created on the remote file system. This + * can fail if the remote file system does not support UDS file types + * (e.g. smbfs to a Windows server) or if the remote kernel does not + * allow a non-local process to bind() the socket. (These problems + * could be fixed by moving the UDS out of the .git directory and to a + * well-known local directory on the client machine, but care should + * be taken to ensure that $HOME is actually local and not a managed + * file share.) + * + * FAT32 and NTFS working directories are problematic too. + * + * The builtin FSMonitor uses a Unix domain socket in the .git + * directory for IPC. These Windows drive formats do not support + * Unix domain sockets, so mark them as incompatible for the daemon. + */ +static enum fsmonitor_reason check_uds_volume(struct repository *r) +{ + struct fs_info fs; + const char *ipc_path = fsmonitor_ipc__get_path(r); + char *path; + char *dir; + + /* + * Create a copy for dirname() since it may modify its argument. + */ + path = xstrdup(ipc_path); + dir = dirname(path); + + if (fsmonitor__get_fs_info(dir, &fs) == -1) { + free(path); + return FSMONITOR_REASON_ERROR; + } + + free(path); + + if (fs.is_remote || + !strcmp(fs.typename, "msdos") || + !strcmp(fs.typename, "ntfs") || + !strcmp(fs.typename, "vfat")) { + free(fs.typename); + return FSMONITOR_REASON_NOSOCKETS; + } + + free(fs.typename); + return FSMONITOR_REASON_OK; +} + +enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc) +{ + enum fsmonitor_reason reason; + + if (ipc) { + reason = check_uds_volume(r); + if (reason != FSMONITOR_REASON_OK) + return reason; + } + + return FSMONITOR_REASON_OK; +} diff --git a/config.mak.uname b/config.mak.uname index 1691c6ae6e..a0fd0752a3 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = linux + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..141b05acb0 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -308,6 +308,16 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + add_compile_definitions(HAVE_LINUX_MAGIC_H) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c) + + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-linux.c) endif() endif() base-commit: 7c7698a654a7a0031f65b0ab0c1c4e438e95df60 -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v3] fsmonitor: implement filesystem change listener for Linux 2025-12-30 12:08 ` [PATCH v2] " Paul Tarjan via GitGitGadget @ 2025-12-30 12:55 ` Paul Tarjan via GitGitGadget 2025-12-31 17:41 ` [PATCH v4] " Paul Tarjan via GitGitGadget 2025-12-30 15:37 ` [PATCH v2] " Junio C Hamano 1 sibling, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2025-12-30 12:55 UTC (permalink / raw) To: git; +Cc: Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring - Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing - Handles directory creation, deletion, and renames dynamically - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote - Creates batches lazily (only for actual file events, not cookies) to avoid spurious sequence number increments Build configuration: - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset - Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Testing performed: - Build succeeds with standard flags and SANITIZE=address - All t7527-builtin-fsmonitor.sh tests pass on local filesystems - Remote filesystem detection correctly rejects network mounts Issues addressed from PR #1352 (git/git) review comments: - GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) - Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup - Unsafe hashmap iteration in dtor: Collect entries first, then modify - Missing null checks in stop_async: Added proper guard conditions - dirname() modifying argument: Create copy with xstrdup() first - Non-portable f_fsid.__val: Use memcmp() for fsid comparison - Missing worktree null check: Added BUG() for null worktree - Header updates: Use git-compat-util.h, hash_to_hex_algop() - Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: - EINTR handling: read() now handles both EAGAIN and EINTR - Trailing pipe in log_mask_set: Added strbuf_strip_suffix() - Unchecked add_watch return: Now logs failure in rename_dir() - String building: Consolidated strbuf operations with strbuf_addf() - Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- fsmonitor: implement filesystem change listener for Linux Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: * Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring * Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing * Handles directory creation, deletion, and renames dynamically * Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() * Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote Build configuration: * Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux * Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset * Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Testing performed: * Build succeeds with standard flags (NO_CURL, NO_GETTEXT, NO_TCLTK) * Build succeeds with SANITIZE=address (AddressSanitizer) * General tests pass (t0001-init, t3200-branch, etc.) * fsmonitor tests require local filesystem (v9fs test environment is correctly detected as remote and rejected) * Manual testing confirms daemon starts and monitors filesystem when configured with fsmonitor.allowremote=true and fsmonitor.socketdir pointing to a local filesystem (tmpfs) Issues addressed from PR #1352 (git/git) review comments: * GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) * Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup * Unsafe hashmap iteration in dtor: Collect entries first, then modify * Missing null checks in stop_async: Added proper guard conditions * dirname() modifying argument: Create copy with xstrdup() first * Non-portable f_fsid.__val: Use memcmp() for fsid comparison * Missing worktree null check: Added BUG() for null worktree * Header updates: Use git-compat-util.h, hash_to_hex_algop() * Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: * EINTR handling: read() now handles both EAGAIN and EINTR * Trailing pipe in log_mask_set: Added strbuf_strip_suffix() * Unchecked add_watch return: Now logs failure in rename_dir() * String building: Consolidated strbuf operations with strbuf_addf() * Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v2: * Included fix khash memory leak in do_handle_client https://github.com/git/git/pull/2148 Changes since v1: * Fixed memory leak in builtin/fsmonitor--daemon.c: Added hashmap_clear(&state.cookies) in the cleanup section of fsmonitor_run_daemon() to free the cookies hashmap that was initialized but never released. This fixes the 512-byte leak detected by LeakSanitizer. * Removed 9p (Plan 9) from remote filesystem list in compat/fsmonitor/fsm-path-utils-linux.c: Removed V9FS_MAGIC from is_remote_fs() because inotify works correctly on 9p filesystems. This was incorrectly preventing fsmonitor from running in 9p-based container environments where inotify is fully functional. Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v3 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v3 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v2: 1: 7a7fe25a06 ! 1: 207da682f4 fsmonitor: implement filesystem change listener for Linux @@ Documentation/git-fsmonitor--daemon.adoc: By default, the socket is created in t ## builtin/fsmonitor--daemon.c ## +@@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state, + const struct fsmonitor_batch *batch; + struct fsmonitor_batch *remainder = NULL; + intmax_t count = 0, duplicates = 0; +- kh_str_t *shown; ++ kh_str_t *shown = NULL; + int hash_ret; + int do_trivial = 0; + int do_flush = 0; +@@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state, + total_response_len += payload.len; + } + +- kh_release_str(shown); +- + pthread_mutex_lock(&state->main_lock); + + if (token_data->client_ref_count > 0) +@@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state, + trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); + + cleanup: ++ kh_destroy_str(shown); + strbuf_release(&response_token); + strbuf_release(&requested_token_id); + strbuf_release(&payload); @@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- builtin/fsmonitor--daemon.c | 6 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-ipc-linux.c | 61 ++ compat/fsmonitor/fsm-listen-linux.c | 738 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 223 ++++++ compat/fsmonitor/fsm-settings-linux.c | 71 ++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 10 + 10 files changed, 1175 insertions(+), 9 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-ipc-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c create mode 100644 compat/fsmonitor/fsm-settings-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..4d52622e24 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); @@ -1405,6 +1404,7 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + hashmap_clear(&state.cookies); fsm_listen__dtor(&state); fsm_health__dtor(&state); diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-ipc-linux.c b/compat/fsmonitor/fsm-ipc-linux.c new file mode 100644 index 0000000000..d34a6419bc --- /dev/null +++ b/compat/fsmonitor/fsm-ipc-linux.c @@ -0,0 +1,61 @@ +#define USE_THE_REPOSITORY_VARIABLE + +#include "git-compat-util.h" +#include "config.h" +#include "gettext.h" +#include "hex.h" +#include "path.h" +#include "repository.h" +#include "strbuf.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" + +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path(struct repository *r) +{ + static const char *ipc_path = NULL; + git_SHA_CTX sha1ctx; + char *sock_dir = NULL; + struct strbuf ipc_file = STRBUF_INIT; + unsigned char hash[GIT_SHA1_RAWSZ]; + + if (!r) + BUG("No repository passed into fsmonitor_ipc__get_path"); + + if (ipc_path) + return ipc_path; + + /* By default the socket file is created in the .git directory */ + if (fsmonitor__is_fs_remote(r->gitdir) < 1) { + ipc_path = fsmonitor_ipc__get_default_path(); + return ipc_path; + } + + if (!r->worktree) + BUG("repository has no worktree"); + + git_SHA1_Init(&sha1ctx); + git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); + git_SHA1_Final(hash, &sha1ctx); + + repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir); + + /* Create the socket file in either socketDir or $HOME */ + if (sock_dir && *sock_dir) { + strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s", + sock_dir, hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } else { + strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", + hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } + free(sock_dir); + + ipc_path = interpolate_path(ipc_file.buf, 1); + if (!ipc_path) + die(_("Invalid path: %s"), ipc_file.buf); + + strbuf_release(&ipc_file); + return ipc_path; +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..2593c834d3 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,738 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <dirent.h> +#include <fcntl.h> +#include <poll.h> +#include <sys/inotify.h> +#include <sys/stat.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("Double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("Removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("Double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("Forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + const int interval = 1000; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 1); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "Missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..fc4acbc20d --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,223 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <errno.h> +#include <stdio.h> +#include <string.h> +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef V9FS_MAGIC +#define V9FS_MAGIC 0x01021997 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Get the filesystem type name for logging purposes. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case V9FS_MAGIC: + return "9p"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * Returns the filesystem type for the longest matching mount point. + */ +static char *find_mount(const char *path, struct statfs *fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + struct statfs path_fs; + + if (statfs(path, &path_fs) < 0) + return NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + *fs = mount_fs; + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/compat/fsmonitor/fsm-settings-linux.c b/compat/fsmonitor/fsm-settings-linux.c new file mode 100644 index 0000000000..23e7442d0c --- /dev/null +++ b/compat/fsmonitor/fsm-settings-linux.c @@ -0,0 +1,71 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-settings.h" +#include "fsmonitor-path-utils.h" + +#include <libgen.h> + +/* + * For the builtin FSMonitor, we create the Unix domain socket for the + * IPC in the .git directory. If the working directory is remote, + * then the socket will be created on the remote file system. This + * can fail if the remote file system does not support UDS file types + * (e.g. smbfs to a Windows server) or if the remote kernel does not + * allow a non-local process to bind() the socket. (These problems + * could be fixed by moving the UDS out of the .git directory and to a + * well-known local directory on the client machine, but care should + * be taken to ensure that $HOME is actually local and not a managed + * file share.) + * + * FAT32 and NTFS working directories are problematic too. + * + * The builtin FSMonitor uses a Unix domain socket in the .git + * directory for IPC. These Windows drive formats do not support + * Unix domain sockets, so mark them as incompatible for the daemon. + */ +static enum fsmonitor_reason check_uds_volume(struct repository *r) +{ + struct fs_info fs; + const char *ipc_path = fsmonitor_ipc__get_path(r); + char *path; + char *dir; + + /* + * Create a copy for dirname() since it may modify its argument. + */ + path = xstrdup(ipc_path); + dir = dirname(path); + + if (fsmonitor__get_fs_info(dir, &fs) == -1) { + free(path); + return FSMONITOR_REASON_ERROR; + } + + free(path); + + if (fs.is_remote || + !strcmp(fs.typename, "msdos") || + !strcmp(fs.typename, "ntfs") || + !strcmp(fs.typename, "vfat")) { + free(fs.typename); + return FSMONITOR_REASON_NOSOCKETS; + } + + free(fs.typename); + return FSMONITOR_REASON_OK; +} + +enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc) +{ + enum fsmonitor_reason reason; + + if (ipc) { + reason = check_uds_volume(r); + if (reason != FSMONITOR_REASON_OK) + return reason; + } + + return FSMONITOR_REASON_OK; +} diff --git a/config.mak.uname b/config.mak.uname index 1691c6ae6e..a0fd0752a3 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = linux + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..141b05acb0 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -308,6 +308,16 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + add_compile_definitions(HAVE_LINUX_MAGIC_H) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c) + + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-linux.c) endif() endif() base-commit: 7c7698a654a7a0031f65b0ab0c1c4e438e95df60 -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2025-12-30 12:55 ` [PATCH v3] " Paul Tarjan via GitGitGadget @ 2025-12-31 17:41 ` Paul Tarjan via GitGitGadget 2026-01-05 12:07 ` Patrick Steinhardt 2026-02-24 1:31 ` [PATCH v5] " Paul Tarjan via GitGitGadget 0 siblings, 2 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2025-12-31 17:41 UTC (permalink / raw) To: git; +Cc: Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring - Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing - Handles directory creation, deletion, and renames dynamically - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote - Creates batches lazily (only for actual file events, not cookies) to avoid spurious sequence number increments Build configuration: - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset - Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Testing performed: - Build succeeds with standard flags and SANITIZE=address - All t7527-builtin-fsmonitor.sh tests pass on local filesystems - Remote filesystem detection correctly rejects network mounts Issues addressed from PR #1352 (git/git) review comments: - GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) - Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup - Unsafe hashmap iteration in dtor: Collect entries first, then modify - Missing null checks in stop_async: Added proper guard conditions - dirname() modifying argument: Create copy with xstrdup() first - Non-portable f_fsid.__val: Use memcmp() for fsid comparison - Missing worktree null check: Added BUG() for null worktree - Header updates: Use git-compat-util.h, hash_to_hex_algop() - Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: - EINTR handling: read() now handles both EAGAIN and EINTR - Trailing pipe in log_mask_set: Added strbuf_strip_suffix() - Unchecked add_watch return: Now logs failure in rename_dir() - String building: Consolidated strbuf operations with strbuf_addf() - Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- fsmonitor: implement filesystem change listener for Linux Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: * Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring * Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing * Handles directory creation, deletion, and renames dynamically * Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() * Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote Build configuration: * Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux * Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset * Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Testing performed: * Build succeeds with standard flags * Build succeeds with SANITIZE=address (AddressSanitizer) * General tests pass (t0001-init, t3200-branch, etc.) Stress testing on Linux (tmpfs): * Parallel file creation: 50,000 files created in parallel batches, all detected correctly * Rapid file deletion: Mass deletion of 10,000+ files tracked properly * Deep nesting: 100-level deep directory hierarchies handled correctly * Directory renames: 2,000+ rename operations including rapid rename chains * Directory move in/out: Directories moved from outside to inside watch tree and back * Concurrent git operations: 100 parallel git status commands while filesystem changes occur * Burst events: 60,000+ rapid-fire inotify events (exceeding default queue size of 16,384) * Race conditions: Rapid mkdir+create+rmdir cycles and rename back-and-forth stress * Sustained load: 60 seconds of continuous high-throughput operations (10 parallel workers), daemon remained stable with 377,000+ tracked files * Git worktrees: 10 worktrees with 500 files across them * Special files: Hardlinks, symlinks, FIFOs (named pipes) * File operations: Permission changes (3,000 chmod), timestamp updates (2,000 touch), truncate operations (3,000 truncates) * Large files: 3x 100MB files created in parallel * Unicode filenames: CJK, Cyrillic, Greek, Arabic, emoji characters * IDE simulation: 1,000 rapid edit/save cycles mimicking editor behavior * Maximum chaos: All operation types running simultaneously Issues addressed from PR #1352 (git/git) review comments: * GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) * Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup * Unsafe hashmap iteration in dtor: Collect entries first, then modify * Missing null checks in stop_async: Added proper guard conditions * dirname() modifying argument: Create copy with xstrdup() first * Non-portable f_fsid.__val: Use memcmp() for fsid comparison * Missing worktree null check: Added BUG() for null worktree * Header updates: Use git-compat-util.h, hash_to_hex_algop() * Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: * EINTR handling: read() now handles both EAGAIN and EINTR * Trailing pipe in log_mask_set: Added strbuf_strip_suffix() * Unchecked add_watch return: Now logs failure in rename_dir() * String building: Consolidated strbuf operations with strbuf_addf() * Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v3: * Fix for rapid creation of nested directories where the daemon could crash with: inotify_add_watch('...') failed: File exists * Did a lot of stress testing Changes since v2: * Included fix khash memory leak in do_handle_client https://github.com/git/git/pull/2148 Changes since v1: * Fixed memory leak in builtin/fsmonitor--daemon.c: Added hashmap_clear(&state.cookies) in the cleanup section of fsmonitor_run_daemon() to free the cookies hashmap that was initialized but never released. This fixes the 512-byte leak detected by LeakSanitizer. * Removed 9p (Plan 9) from remote filesystem list in compat/fsmonitor/fsm-path-utils-linux.c: Removed V9FS_MAGIC from is_remote_fs() because inotify works correctly on 9p filesystems. This was incorrectly preventing fsmonitor from running in 9p-based container environments where inotify is fully functional. Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v4 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v4 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v3: 1: 207da682f4 ! 1: c7fd346e89 fsmonitor: implement filesystem change listener for Linux @@ compat/fsmonitor/fsm-listen-linux.c (new) + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ ++ if (errno == EEXIST) ++ return 0; /* watch already exists, no action needed */ + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) endif() endif() + + ## t/t7527-builtin-fsmonitor.sh ## +@@ t/t7527-builtin-fsmonitor.sh: test_expect_success 'directory changes to a file' ' + grep "^event: dir1$" .git/trace + ' + ++test_expect_success 'rapid nested directory creation' ' ++ test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && ++ ++ start_daemon --tf "$PWD/.git/trace" && ++ ++ # Rapidly create nested directories to exercise race conditions ++ # where directory watches may be added concurrently during ++ # event processing and recursive scanning. ++ for i in $(test_seq 1 20) ++ do ++ mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 ++ done && ++ ++ # Give the daemon time to process all events ++ sleep 1 && ++ ++ test-tool fsmonitor-client query --token 0 && ++ ++ # Verify daemon is still running (did not crash) ++ git fsmonitor--daemon status ++' ++ + # The next few test cases exercise the token-resync code. When filesystem + # drops events (because of filesystem velocity or because the daemon isn't + # polling fast enough), we need to discard the cached data (relative to the Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- builtin/fsmonitor--daemon.c | 6 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-ipc-linux.c | 61 ++ compat/fsmonitor/fsm-listen-linux.c | 740 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 223 ++++++ compat/fsmonitor/fsm-settings-linux.c | 71 ++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 10 + t/t7527-builtin-fsmonitor.sh | 22 + 11 files changed, 1199 insertions(+), 9 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-ipc-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c create mode 100644 compat/fsmonitor/fsm-settings-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..4d52622e24 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); @@ -1405,6 +1404,7 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + hashmap_clear(&state.cookies); fsm_listen__dtor(&state); fsm_health__dtor(&state); diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-ipc-linux.c b/compat/fsmonitor/fsm-ipc-linux.c new file mode 100644 index 0000000000..d34a6419bc --- /dev/null +++ b/compat/fsmonitor/fsm-ipc-linux.c @@ -0,0 +1,61 @@ +#define USE_THE_REPOSITORY_VARIABLE + +#include "git-compat-util.h" +#include "config.h" +#include "gettext.h" +#include "hex.h" +#include "path.h" +#include "repository.h" +#include "strbuf.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" + +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path(struct repository *r) +{ + static const char *ipc_path = NULL; + git_SHA_CTX sha1ctx; + char *sock_dir = NULL; + struct strbuf ipc_file = STRBUF_INIT; + unsigned char hash[GIT_SHA1_RAWSZ]; + + if (!r) + BUG("No repository passed into fsmonitor_ipc__get_path"); + + if (ipc_path) + return ipc_path; + + /* By default the socket file is created in the .git directory */ + if (fsmonitor__is_fs_remote(r->gitdir) < 1) { + ipc_path = fsmonitor_ipc__get_default_path(); + return ipc_path; + } + + if (!r->worktree) + BUG("repository has no worktree"); + + git_SHA1_Init(&sha1ctx); + git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); + git_SHA1_Final(hash, &sha1ctx); + + repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir); + + /* Create the socket file in either socketDir or $HOME */ + if (sock_dir && *sock_dir) { + strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s", + sock_dir, hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } else { + strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", + hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } + free(sock_dir); + + ipc_path = interpolate_path(ipc_file.buf, 1); + if (!ipc_path) + die(_("Invalid path: %s"), ipc_file.buf); + + strbuf_release(&ipc_file); + return ipc_path; +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..04441c5120 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,740 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <dirent.h> +#include <fcntl.h> +#include <poll.h> +#include <sys/inotify.h> +#include <sys/stat.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("Double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("Removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("Double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("Forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + const int interval = 1000; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 1); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "Missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..fc4acbc20d --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,223 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <errno.h> +#include <stdio.h> +#include <string.h> +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef V9FS_MAGIC +#define V9FS_MAGIC 0x01021997 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Get the filesystem type name for logging purposes. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case V9FS_MAGIC: + return "9p"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * Returns the filesystem type for the longest matching mount point. + */ +static char *find_mount(const char *path, struct statfs *fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + struct statfs path_fs; + + if (statfs(path, &path_fs) < 0) + return NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + *fs = mount_fs; + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/compat/fsmonitor/fsm-settings-linux.c b/compat/fsmonitor/fsm-settings-linux.c new file mode 100644 index 0000000000..23e7442d0c --- /dev/null +++ b/compat/fsmonitor/fsm-settings-linux.c @@ -0,0 +1,71 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-settings.h" +#include "fsmonitor-path-utils.h" + +#include <libgen.h> + +/* + * For the builtin FSMonitor, we create the Unix domain socket for the + * IPC in the .git directory. If the working directory is remote, + * then the socket will be created on the remote file system. This + * can fail if the remote file system does not support UDS file types + * (e.g. smbfs to a Windows server) or if the remote kernel does not + * allow a non-local process to bind() the socket. (These problems + * could be fixed by moving the UDS out of the .git directory and to a + * well-known local directory on the client machine, but care should + * be taken to ensure that $HOME is actually local and not a managed + * file share.) + * + * FAT32 and NTFS working directories are problematic too. + * + * The builtin FSMonitor uses a Unix domain socket in the .git + * directory for IPC. These Windows drive formats do not support + * Unix domain sockets, so mark them as incompatible for the daemon. + */ +static enum fsmonitor_reason check_uds_volume(struct repository *r) +{ + struct fs_info fs; + const char *ipc_path = fsmonitor_ipc__get_path(r); + char *path; + char *dir; + + /* + * Create a copy for dirname() since it may modify its argument. + */ + path = xstrdup(ipc_path); + dir = dirname(path); + + if (fsmonitor__get_fs_info(dir, &fs) == -1) { + free(path); + return FSMONITOR_REASON_ERROR; + } + + free(path); + + if (fs.is_remote || + !strcmp(fs.typename, "msdos") || + !strcmp(fs.typename, "ntfs") || + !strcmp(fs.typename, "vfat")) { + free(fs.typename); + return FSMONITOR_REASON_NOSOCKETS; + } + + free(fs.typename); + return FSMONITOR_REASON_OK; +} + +enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc) +{ + enum fsmonitor_reason reason; + + if (ipc) { + reason = check_uds_volume(r); + if (reason != FSMONITOR_REASON_OK) + return reason; + } + + return FSMONITOR_REASON_OK; +} diff --git a/config.mak.uname b/config.mak.uname index 1691c6ae6e..a0fd0752a3 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = linux + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..141b05acb0 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -308,6 +308,16 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + add_compile_definitions(HAVE_LINUX_MAGIC_H) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c) + + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-linux.c) endif() endif() diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..d2f1f1097e 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -520,6 +520,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the base-commit: 7c7698a654a7a0031f65b0ab0c1c4e438e95df60 -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2025-12-31 17:41 ` [PATCH v4] " Paul Tarjan via GitGitGadget @ 2026-01-05 12:07 ` Patrick Steinhardt 2026-02-20 22:18 ` Junio C Hamano 2026-02-24 1:31 ` [PATCH v5] " Paul Tarjan via GitGitGadget 1 sibling, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-01-05 12:07 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Wed, Dec 31, 2025 at 05:41:34PM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> > > Implement fsmonitor for Linux using the inotify API, bringing it to > feature parity with existing Windows and macOS implementations. > > The Linux implementation uses inotify to monitor filesystem events. > Unlike macOS's FSEvents which can watch a single root directory, > inotify requires registering watches on every directory of interest. > The implementation carefully handles directory renames and moves > using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO > event pairs. > > Key implementation details: > - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring > - Maintains bidirectional hashmaps between watch descriptors and paths > for efficient event processing > - Handles directory creation, deletion, and renames dynamically > - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() > - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote > - Creates batches lazily (only for actual file events, not cookies) > to avoid spurious sequence number increments > > Build configuration: > - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux > - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset > - Adds HAVE_LINUX_MAGIC_H for filesystem type detection This would also need the below patch to support Meson. Would be great if you include it, otherwise I can send it as a separate patch once this topic lands. Thanks! Patrick -- >8 -- diff --git a/meson.build b/meson.build index dd52efd1c8..0130d40702 100644 --- a/meson.build +++ b/meson.build @@ -1322,6 +1322,9 @@ endif fsmonitor_backend = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' libgit_dependencies += dependency('CoreServices') ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-01-05 12:07 ` Patrick Steinhardt @ 2026-02-20 22:18 ` Junio C Hamano 2026-02-21 16:15 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-20 22:18 UTC (permalink / raw) To: Patrick Steinhardt, Paul Tarjan; +Cc: Paul Tarjan via GitGitGadget, git Patrick Steinhardt <ps@pks.im> writes: > This would also need the below patch to support Meson. Would be great if > you include it, otherwise I can send it as a separate patch once this > topic lands. Thanks! > > Patrick > ... I just noticed that the discussion thread went silent after this message. Has the patch been reviewed and tested well to proceed, except for that meson-build support? Thanks. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-02-20 22:18 ` Junio C Hamano @ 2026-02-21 16:15 ` Paul Tarjan 2026-02-21 17:07 ` Junio C Hamano 0 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan @ 2026-02-21 16:15 UTC (permalink / raw) To: Junio C Hamano Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan via GitGitGadget, git > I just noticed that the discussion thread went silent after this > message. Has the patch been reviewed and tested well to proceed, > except for that meson-build support? I'd love to see it integrated upstream. Is there anything else you need from me? I've been using the patch for 2 months at my work over a very large fleet and it has been working great. > > This would also need the below patch to support Meson. Would be great if > > you include it, otherwise I can send it as a separate patch once this > > topic lands. Thanks! I'd prefer to take you up on the offer to send the meson support as a separate patch. I'm unfamiliar with that system and the suggested patch failed in CI on some dependency installation steps which felt unrelated but I didn't want to debug. https://github.com/git/git/actions/runs/20720903513 Thanks Paul ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-02-21 16:15 ` Paul Tarjan @ 2026-02-21 17:07 ` Junio C Hamano 2026-02-23 6:34 ` Patrick Steinhardt 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-21 17:07 UTC (permalink / raw) To: Paul Tarjan Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan via GitGitGadget, git Paul Tarjan <paul@paultarjan.com> writes: >> I just noticed that the discussion thread went silent after this >> message. Has the patch been reviewed and tested well to proceed, >> except for that meson-build support? > > I'd love to see it integrated upstream. Is there anything else you > need from me? > >> > This would also need the below patch to support Meson. Would be great if >> > you include it, otherwise I can send it as a separate patch once this >> > topic lands. Thanks! > > I'd prefer to take you up on the offer to send the meson support as a > separate patch. This part of your message is one thing we needed from you to unblock ourselves, I guess. Patrick, do you think you can help making this into two-patch series, the original one being the [PATCH 1/2] and update for meson-build in [PATCH 2/2]? > I'm unfamiliar with that system and the suggested > patch failed in CI on some dependency installation steps which felt > unrelated but I didn't want to debug. > https://github.com/git/git/actions/runs/20720903513 The topic has been in my tree near the tip of 'seen' and I do not think we saw CI failures coming from this topic. Thanks. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-02-21 17:07 ` Junio C Hamano @ 2026-02-23 6:34 ` Patrick Steinhardt 2026-02-23 15:42 ` Junio C Hamano 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-02-23 6:34 UTC (permalink / raw) To: Junio C Hamano Cc: Paul Tarjan, Paul Tarjan, Paul Tarjan via GitGitGadget, git On Sat, Feb 21, 2026 at 09:07:52AM -0800, Junio C Hamano wrote: > Paul Tarjan <paul@paultarjan.com> writes: > > >> I just noticed that the discussion thread went silent after this > >> message. Has the patch been reviewed and tested well to proceed, > >> except for that meson-build support? > > > > I'd love to see it integrated upstream. Is there anything else you > > need from me? > > > >> > This would also need the below patch to support Meson. Would be great if > >> > you include it, otherwise I can send it as a separate patch once this > >> > topic lands. Thanks! > > > > I'd prefer to take you up on the offer to send the meson support as a > > separate patch. > > This part of your message is one thing we needed from you to unblock > ourselves, I guess. > > Patrick, do you think you can help making this into two-patch > series, the original one being the [PATCH 1/2] and update for > meson-build in [PATCH 2/2]? The changes I sent should be sufficient, so I'd propose to just roll it into the v5 patch. > > I'm unfamiliar with that system and the suggested > > patch failed in CI on some dependency installation steps which felt > > unrelated but I didn't want to debug. > > https://github.com/git/git/actions/runs/20720903513 > > The topic has been in my tree near the tip of 'seen' and I do not > think we saw CI failures coming from this topic. Yeah, I think this was simply a flaky CI job. The "linux-reftable" job has failed installing packages: Err:3 http://security.ubuntu.com/ubuntu questing-security/universe amd64 Packages 404 Not Found [IP: 91.189.91.83 80] File has unexpected size (89328 != 89310). Mirror sync in progress? [IP: 91.189.91.83 80] Hashes of expected file: - Filesize:89310 [weak] - SHA256:16943889a9abc4aaeb0e701e99db0004ac0241de728183f7a2923bb7927b107b - SHA1:fac9e79fb36b57de3770e639c4b5bf9231342f15 [weak] - MD5Sum:8cf081c59fbb279da27867d94f2b9520 [weak] Release file created at: Mon, 05 Jan 2026 13:30:45 +0000 And all the other jobs simply got aborted because of that initial failure. In any case, the changes work alright with Meson on my system. By the way, I haven't yet done a full review of this patch, I only chimed in to help out with Meson. But I can have a deeper look once v5 was sent out. Thanks! Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-02-23 6:34 ` Patrick Steinhardt @ 2026-02-23 15:42 ` Junio C Hamano 2026-02-23 15:46 ` Patrick Steinhardt 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-23 15:42 UTC (permalink / raw) To: Patrick Steinhardt Cc: Paul Tarjan, Paul Tarjan, Paul Tarjan via GitGitGadget, git Patrick Steinhardt <ps@pks.im> writes: > On Sat, Feb 21, 2026 at 09:07:52AM -0800, Junio C Hamano wrote: >> Paul Tarjan <paul@paultarjan.com> writes: >> >> > I'd prefer to take you up on the offer to send the meson support as a >> > separate patch. >> >> This part of your message is one thing we needed from you to unblock >> ourselves, I guess. >> >> Patrick, do you think you can help making this into two-patch >> series, the original one being the [PATCH 1/2] and update for >> meson-build in [PATCH 2/2]? > > The changes I sent should be sufficient, so I'd propose to just roll > it into the v5 patch. > ... > By the way, I haven't yet done a full review of this patch, I only > chimed in to help out with Meson. But I can have a deeper look once v5 > was sent out. OK, so it is not quite clear to me who is doing the v5. Is the "offer to send the meson support as a separate patch" still valid, or we expect Paul to squash in the earlier patch from you to prepare the v5? Thanks. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-02-23 15:42 ` Junio C Hamano @ 2026-02-23 15:46 ` Patrick Steinhardt 2026-02-24 1:34 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-02-23 15:46 UTC (permalink / raw) To: Junio C Hamano Cc: Paul Tarjan, Paul Tarjan, Paul Tarjan via GitGitGadget, git On Mon, Feb 23, 2026 at 07:42:14AM -0800, Junio C Hamano wrote: > Patrick Steinhardt <ps@pks.im> writes: > > > On Sat, Feb 21, 2026 at 09:07:52AM -0800, Junio C Hamano wrote: > >> Paul Tarjan <paul@paultarjan.com> writes: > >> > >> > I'd prefer to take you up on the offer to send the meson support as a > >> > separate patch. > >> > >> This part of your message is one thing we needed from you to unblock > >> ourselves, I guess. > >> > >> Patrick, do you think you can help making this into two-patch > >> series, the original one being the [PATCH 1/2] and update for > >> meson-build in [PATCH 2/2]? > > > > The changes I sent should be sufficient, so I'd propose to just roll > > it into the v5 patch. > > ... > > By the way, I haven't yet done a full review of this patch, I only > > chimed in to help out with Meson. But I can have a deeper look once v5 > > was sent out. > > OK, so it is not quite clear to me who is doing the v5. Is the > "offer to send the meson support as a separate patch" still valid, > or we expect Paul to squash in the earlier patch from you to prepare > the v5? I'd think the latter, Paul squashes my patch into his commit. I don't think it needs to be a separate patch. Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-02-23 15:46 ` Patrick Steinhardt @ 2026-02-24 1:34 ` Paul Tarjan 2026-02-24 8:03 ` Patrick Steinhardt 0 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan @ 2026-02-24 1:34 UTC (permalink / raw) To: Patrick Steinhardt Cc: Junio C Hamano, Paul Tarjan, Paul Tarjan via GitGitGadget, git On Mon, Feb 23, 2026 at 5:01 PM Patrick Steinhardt <ps@pks.im> wrote: > > On Mon, Feb 23, 2026 at 07:42:14AM -0800, Junio C Hamano wrote: > > Patrick Steinhardt <ps@pks.im> writes: > > > > > On Sat, Feb 21, 2026 at 09:07:52AM -0800, Junio C Hamano wrote: > > >> Paul Tarjan <paul@paultarjan.com> writes: > > >> > > >> > I'd prefer to take you up on the offer to send the meson support as a > > >> > separate patch. > > >> > > >> This part of your message is one thing we needed from you to unblock > > >> ourselves, I guess. > > >> > > >> Patrick, do you think you can help making this into two-patch > > >> series, the original one being the [PATCH 1/2] and update for > > >> meson-build in [PATCH 2/2]? > > > > > > The changes I sent should be sufficient, so I'd propose to just roll > > > it into the v5 patch. > > > ... > > > By the way, I haven't yet done a full review of this patch, I only > > > chimed in to help out with Meson. But I can have a deeper look once v5 > > > was sent out. > > > > OK, so it is not quite clear to me who is doing the v5. Is the > > "offer to send the meson support as a separate patch" still valid, > > or we expect Paul to squash in the earlier patch from you to prepare > > the v5? > > I'd think the latter, Paul squashes my patch into his commit. I don't > think it needs to be a separate patch. Done. CI still failed in a different way this time: https://github.com/git/git/actions/runs/22311198802/job/64543526340?pr=2147 ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v4] fsmonitor: implement filesystem change listener for Linux 2026-02-24 1:34 ` Paul Tarjan @ 2026-02-24 8:03 ` Patrick Steinhardt 0 siblings, 0 replies; 129+ messages in thread From: Patrick Steinhardt @ 2026-02-24 8:03 UTC (permalink / raw) To: Paul Tarjan Cc: Junio C Hamano, Paul Tarjan, Paul Tarjan via GitGitGadget, git On Mon, Feb 23, 2026 at 06:34:31PM -0700, Paul Tarjan wrote: > On Mon, Feb 23, 2026 at 5:01 PM Patrick Steinhardt <ps@pks.im> wrote: > > > > On Mon, Feb 23, 2026 at 07:42:14AM -0800, Junio C Hamano wrote: > > > Patrick Steinhardt <ps@pks.im> writes: > > > > > > > On Sat, Feb 21, 2026 at 09:07:52AM -0800, Junio C Hamano wrote: > > > >> Paul Tarjan <paul@paultarjan.com> writes: > > > >> > > > >> > I'd prefer to take you up on the offer to send the meson support as a > > > >> > separate patch. > > > >> > > > >> This part of your message is one thing we needed from you to unblock > > > >> ourselves, I guess. > > > >> > > > >> Patrick, do you think you can help making this into two-patch > > > >> series, the original one being the [PATCH 1/2] and update for > > > >> meson-build in [PATCH 2/2]? > > > > > > > > The changes I sent should be sufficient, so I'd propose to just roll > > > > it into the v5 patch. > > > > ... > > > > By the way, I haven't yet done a full review of this patch, I only > > > > chimed in to help out with Meson. But I can have a deeper look once v5 > > > > was sent out. > > > > > > OK, so it is not quite clear to me who is doing the v5. Is the > > > "offer to send the meson support as a separate patch" still valid, > > > or we expect Paul to squash in the earlier patch from you to prepare > > > the v5? > > > > I'd think the latter, Paul squashes my patch into his commit. I don't > > think it needs to be a separate patch. > > Done. CI still failed in a different way this time: > https://github.com/git/git/actions/runs/22311198802/job/64543526340?pr=2147 This looks like executing tests got stuck. Makes me wonder whether there is maybe a race condition in the new fsmonitor implementation that causes tests to not progress anymore. Let me have a deeper look at the code. Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v5] fsmonitor: implement filesystem change listener for Linux 2025-12-31 17:41 ` [PATCH v4] " Paul Tarjan via GitGitGadget 2026-01-05 12:07 ` Patrick Steinhardt @ 2026-02-24 1:31 ` Paul Tarjan via GitGitGadget 2026-02-24 8:03 ` Patrick Steinhardt 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget 1 sibling, 2 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-24 1:31 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring - Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing - Handles directory creation, deletion, and renames dynamically - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote Build configuration: - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset - Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Issues addressed from PR #1352 (git/git) review comments: - GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) - Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup - Unsafe hashmap iteration in dtor: Collect entries first, then modify - Missing null checks in stop_async: Added proper guard conditions - dirname() modifying argument: Create copy with xstrdup() first - Non-portable f_fsid.__val: Use memcmp() for fsid comparison - Missing worktree null check: Added BUG() for null worktree - Header updates: Use git-compat-util.h, hash_to_hex_algop() - Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: - EINTR handling: read() now handles both EAGAIN and EINTR - Trailing pipe in log_mask_set: Added strbuf_strip_suffix() - Unchecked add_watch return: Now logs failure in rename_dir() - String building: Consolidated strbuf operations with strbuf_addf() - Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- fsmonitor: implement filesystem change listener for Linux Implement fsmonitor for Linux using the inotify API, bringing it to feature parity with existing Windows and macOS implementations. The Linux implementation uses inotify to monitor filesystem events. Unlike macOS's FSEvents which can watch a single root directory, inotify requires registering watches on every directory of interest. The implementation carefully handles directory renames and moves using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO event pairs. Key implementation details: * Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring * Maintains bidirectional hashmaps between watch descriptors and paths for efficient event processing * Handles directory creation, deletion, and renames dynamically * Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() * Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote Build configuration: * Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux * Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset * Adds HAVE_LINUX_MAGIC_H for filesystem type detection Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. Testing performed: * Build succeeds with standard flags * Build succeeds with SANITIZE=address (AddressSanitizer) * General tests pass (t0001-init, t3200-branch, etc.) Stress testing on Linux (tmpfs): * Parallel file creation: 50,000 files created in parallel batches, all detected correctly * Rapid file deletion: Mass deletion of 10,000+ files tracked properly * Deep nesting: 100-level deep directory hierarchies handled correctly * Directory renames: 2,000+ rename operations including rapid rename chains * Directory move in/out: Directories moved from outside to inside watch tree and back * Concurrent git operations: 100 parallel git status commands while filesystem changes occur * Burst events: 60,000+ rapid-fire inotify events (exceeding default queue size of 16,384) * Race conditions: Rapid mkdir+create+rmdir cycles and rename back-and-forth stress * Sustained load: 60 seconds of continuous high-throughput operations (10 parallel workers), daemon remained stable with 377,000+ tracked files * Git worktrees: 10 worktrees with 500 files across them * Special files: Hardlinks, symlinks, FIFOs (named pipes) * File operations: Permission changes (3,000 chmod), timestamp updates (2,000 touch), truncate operations (3,000 truncates) * Large files: 3x 100MB files created in parallel * Unicode filenames: CJK, Cyrillic, Greek, Arabic, emoji characters * IDE simulation: 1,000 rapid edit/save cycles mimicking editor behavior * Maximum chaos: All operation types running simultaneously Issues addressed from PR #1352 (git/git) review comments: * GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) * Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup * Unsafe hashmap iteration in dtor: Collect entries first, then modify * Missing null checks in stop_async: Added proper guard conditions * dirname() modifying argument: Create copy with xstrdup() first * Non-portable f_fsid.__val: Use memcmp() for fsid comparison * Missing worktree null check: Added BUG() for null worktree * Header updates: Use git-compat-util.h, hash_to_hex_algop() * Code style: Use xstrdup() not xmemdupz(), proper pointer style Issues addressed from PR #1667 (git/git) review comments: * EINTR handling: read() now handles both EAGAIN and EINTR * Trailing pipe in log_mask_set: Added strbuf_strip_suffix() * Unchecked add_watch return: Now logs failure in rename_dir() * String building: Consolidated strbuf operations with strbuf_addf() * Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v3: * Fix for rapid creation of nested directories where the daemon could crash with: inotify_add_watch('...') failed: File exists * Did a lot of stress testing Changes since v2: * Included fix khash memory leak in do_handle_client https://github.com/git/git/pull/2148 Changes since v1: * Fixed memory leak in builtin/fsmonitor--daemon.c: Added hashmap_clear(&state.cookies) in the cleanup section of fsmonitor_run_daemon() to free the cookies hashmap that was initialized but never released. This fixes the 512-byte leak detected by LeakSanitizer. * Removed 9p (Plan 9) from remote filesystem list in compat/fsmonitor/fsm-path-utils-linux.c: Removed V9FS_MAGIC from is_remote_fs() because inotify works correctly on 9p filesystems. This was incorrectly preventing fsmonitor from running in 9p-based container environments where inotify is fully functional. Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v5 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v5 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v4: 1: c7fd346e89 ! 1: c54814eb31 fsmonitor: implement filesystem change listener for Linux @@ Commit message - Handles directory creation, deletion, and renames dynamically - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote - - Creates batches lazily (only for actual file events, not cookies) - to avoid spurious sequence number increments Build configuration: - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux @@ Commit message Documentation updated to note that fsmonitor.socketDir is now supported on both Mac OS and Linux, and adds a section about inotify watch limits. - Testing performed: - - Build succeeds with standard flags and SANITIZE=address - - All t7527-builtin-fsmonitor.sh tests pass on local filesystems - - Remote filesystem detection correctly rejects network mounts - Issues addressed from PR #1352 (git/git) review comments: - GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from scratch using statfs() and linux/magic.h constants (no GPLv3 code) @@ Commit message - Translation markers: Added _() to all error_errno() messages Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, - and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated - to work with the current codebase and address all review feedback. + and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to + work with the current codebase and address all review feedback. Signed-off-by: Paul Tarjan <github@paulisageek.com> @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) endif() + ## meson.build ## +@@ meson.build: endif + fsmonitor_backend = '' + if host_machine.system() == 'windows' + fsmonitor_backend = 'win32' ++elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') ++ fsmonitor_backend = 'linux' ++ libgit_c_args += '-DHAVE_LINUX_MAGIC_H' + elif host_machine.system() == 'darwin' + fsmonitor_backend = 'darwin' + libgit_dependencies += dependency('CoreServices') + ## t/t7527-builtin-fsmonitor.sh ## @@ t/t7527-builtin-fsmonitor.sh: test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- builtin/fsmonitor--daemon.c | 6 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-ipc-linux.c | 61 ++ compat/fsmonitor/fsm-listen-linux.c | 740 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 223 ++++++ compat/fsmonitor/fsm-settings-linux.c | 71 ++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 10 + meson.build | 3 + t/t7527-builtin-fsmonitor.sh | 22 + 12 files changed, 1202 insertions(+), 9 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-ipc-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c create mode 100644 compat/fsmonitor/fsm-settings-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..4d52622e24 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); @@ -1405,6 +1404,7 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + hashmap_clear(&state.cookies); fsm_listen__dtor(&state); fsm_health__dtor(&state); diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-ipc-linux.c b/compat/fsmonitor/fsm-ipc-linux.c new file mode 100644 index 0000000000..d34a6419bc --- /dev/null +++ b/compat/fsmonitor/fsm-ipc-linux.c @@ -0,0 +1,61 @@ +#define USE_THE_REPOSITORY_VARIABLE + +#include "git-compat-util.h" +#include "config.h" +#include "gettext.h" +#include "hex.h" +#include "path.h" +#include "repository.h" +#include "strbuf.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-path-utils.h" + +static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") + +const char *fsmonitor_ipc__get_path(struct repository *r) +{ + static const char *ipc_path = NULL; + git_SHA_CTX sha1ctx; + char *sock_dir = NULL; + struct strbuf ipc_file = STRBUF_INIT; + unsigned char hash[GIT_SHA1_RAWSZ]; + + if (!r) + BUG("No repository passed into fsmonitor_ipc__get_path"); + + if (ipc_path) + return ipc_path; + + /* By default the socket file is created in the .git directory */ + if (fsmonitor__is_fs_remote(r->gitdir) < 1) { + ipc_path = fsmonitor_ipc__get_default_path(); + return ipc_path; + } + + if (!r->worktree) + BUG("repository has no worktree"); + + git_SHA1_Init(&sha1ctx); + git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); + git_SHA1_Final(hash, &sha1ctx); + + repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir); + + /* Create the socket file in either socketDir or $HOME */ + if (sock_dir && *sock_dir) { + strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s", + sock_dir, hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } else { + strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", + hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); + } + free(sock_dir); + + ipc_path = interpolate_path(ipc_file.buf, 1); + if (!ipc_path) + die(_("Invalid path: %s"), ipc_file.buf); + + strbuf_release(&ipc_file); + return ipc_path; +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..04441c5120 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,740 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <dirent.h> +#include <fcntl.h> +#include <poll.h> +#include <sys/inotify.h> +#include <sys/stat.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("Double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("Removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("Double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "No matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("Forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + const int interval = 1000; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 1); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "Missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..fc4acbc20d --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,223 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <errno.h> +#include <stdio.h> +#include <string.h> +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef V9FS_MAGIC +#define V9FS_MAGIC 0x01021997 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Get the filesystem type name for logging purposes. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case V9FS_MAGIC: + return "9p"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * Returns the filesystem type for the longest matching mount point. + */ +static char *find_mount(const char *path, struct statfs *fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + struct statfs path_fs; + + if (statfs(path, &path_fs) < 0) + return NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + *fs = mount_fs; + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/compat/fsmonitor/fsm-settings-linux.c b/compat/fsmonitor/fsm-settings-linux.c new file mode 100644 index 0000000000..23e7442d0c --- /dev/null +++ b/compat/fsmonitor/fsm-settings-linux.c @@ -0,0 +1,71 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-ipc.h" +#include "fsmonitor-settings.h" +#include "fsmonitor-path-utils.h" + +#include <libgen.h> + +/* + * For the builtin FSMonitor, we create the Unix domain socket for the + * IPC in the .git directory. If the working directory is remote, + * then the socket will be created on the remote file system. This + * can fail if the remote file system does not support UDS file types + * (e.g. smbfs to a Windows server) or if the remote kernel does not + * allow a non-local process to bind() the socket. (These problems + * could be fixed by moving the UDS out of the .git directory and to a + * well-known local directory on the client machine, but care should + * be taken to ensure that $HOME is actually local and not a managed + * file share.) + * + * FAT32 and NTFS working directories are problematic too. + * + * The builtin FSMonitor uses a Unix domain socket in the .git + * directory for IPC. These Windows drive formats do not support + * Unix domain sockets, so mark them as incompatible for the daemon. + */ +static enum fsmonitor_reason check_uds_volume(struct repository *r) +{ + struct fs_info fs; + const char *ipc_path = fsmonitor_ipc__get_path(r); + char *path; + char *dir; + + /* + * Create a copy for dirname() since it may modify its argument. + */ + path = xstrdup(ipc_path); + dir = dirname(path); + + if (fsmonitor__get_fs_info(dir, &fs) == -1) { + free(path); + return FSMONITOR_REASON_ERROR; + } + + free(path); + + if (fs.is_remote || + !strcmp(fs.typename, "msdos") || + !strcmp(fs.typename, "ntfs") || + !strcmp(fs.typename, "vfat")) { + free(fs.typename); + return FSMONITOR_REASON_NOSOCKETS; + } + + free(fs.typename); + return FSMONITOR_REASON_OK; +} + +enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc) +{ + enum fsmonitor_reason reason; + + if (ipc) { + reason = check_uds_volume(r); + if (reason != FSMONITOR_REASON_OK) + return reason; + } + + return FSMONITOR_REASON_OK; +} diff --git a/config.mak.uname b/config.mak.uname index 1691c6ae6e..a0fd0752a3 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = linux + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..141b05acb0 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -308,6 +308,16 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + add_compile_definitions(HAVE_LINUX_MAGIC_H) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c) + + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-linux.c) endif() endif() diff --git a/meson.build b/meson.build index dd52efd1c8..90e2f77908 100644 --- a/meson.build +++ b/meson.build @@ -1322,6 +1322,9 @@ endif fsmonitor_backend = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' libgit_dependencies += dependency('CoreServices') diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..d2f1f1097e 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -520,6 +520,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the base-commit: 7c7698a654a7a0031f65b0ab0c1c4e438e95df60 -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v5] fsmonitor: implement filesystem change listener for Linux 2026-02-24 1:31 ` [PATCH v5] " Paul Tarjan via GitGitGadget @ 2026-02-24 8:03 ` Patrick Steinhardt 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget 1 sibling, 0 replies; 129+ messages in thread From: Patrick Steinhardt @ 2026-02-24 8:03 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan, Paul Tarjan On Tue, Feb 24, 2026 at 01:31:44AM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> > > Implement fsmonitor for Linux using the inotify API, bringing it to > feature parity with existing Windows and macOS implementations. > > The Linux implementation uses inotify to monitor filesystem events. > Unlike macOS's FSEvents which can watch a single root directory, > inotify requires registering watches on every directory of interest. > The implementation carefully handles directory renames and moves > using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO > event pairs. I think other than stating that this uses inotify, what this commit message should also explain is why it was chosen over its alternatives like fanotify. > Key implementation details: > - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring > - Maintains bidirectional hashmaps between watch descriptors and paths > for efficient event processing > - Handles directory creation, deletion, and renames dynamically > - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() > - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote > > Build configuration: > - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux > - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset > - Adds HAVE_LINUX_MAGIC_H for filesystem type detection It would be great to avoid all these bulleted lists here and instead have some paragraphs that explain the design decisions. > Documentation updated to note that fsmonitor.socketDir is now supported > on both Mac OS and Linux, and adds a section about inotify watch limits. > > Issues addressed from PR #1352 (git/git) review comments: > - GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from > scratch using statfs() and linux/magic.h constants (no GPLv3 code) > - Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup > - Unsafe hashmap iteration in dtor: Collect entries first, then modify > - Missing null checks in stop_async: Added proper guard conditions > - dirname() modifying argument: Create copy with xstrdup() first > - Non-portable f_fsid.__val: Use memcmp() for fsid comparison > - Missing worktree null check: Added BUG() for null worktree > - Header updates: Use git-compat-util.h, hash_to_hex_algop() > - Code style: Use xstrdup() not xmemdupz(), proper pointer style > > Issues addressed from PR #1667 (git/git) review comments: > - EINTR handling: read() now handles both EAGAIN and EINTR > - Trailing pipe in log_mask_set: Added strbuf_strip_suffix() > - Unchecked add_watch return: Now logs failure in rename_dir() > - String building: Consolidated strbuf operations with strbuf_addf() > - Translation markers: Added _() to all error_errno() messages The issues that you've addressed don't typically belong into the commit message. They are useful context when part of a cover letter, but are less useful in the commit messages themselves. > Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, > and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to > work with the current codebase and address all review feedback. In that case it might make sense to say "Based-on-patch-by:" for both of these authors. > diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c > index 242c594646..4d52622e24 100644 > --- a/builtin/fsmonitor--daemon.c > +++ b/builtin/fsmonitor--daemon.c > @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > const struct fsmonitor_batch *batch; > struct fsmonitor_batch *remainder = NULL; > intmax_t count = 0, duplicates = 0; > - kh_str_t *shown; > + kh_str_t *shown = NULL; > int hash_ret; > int do_trivial = 0; > int do_flush = 0; > @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > total_response_len += payload.len; > } > > - kh_release_str(shown); > - > pthread_mutex_lock(&state->main_lock); > > if (token_data->client_ref_count > 0) > @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); > > cleanup: > + kh_destroy_str(shown); > strbuf_release(&response_token); > strbuf_release(&requested_token_id); > strbuf_release(&payload); > @@ -1405,6 +1404,7 @@ static int fsmonitor_run_daemon(void) > done: > pthread_cond_destroy(&state.cookies_cond); > pthread_mutex_destroy(&state.main_lock); > + hashmap_clear(&state.cookies); > fsm_listen__dtor(&state); > fsm_health__dtor(&state); > These feel like while-at-it memory leak fixes. They should probably be moved into separate commits with a proper explanation. > diff --git a/compat/fsmonitor/fsm-ipc-linux.c b/compat/fsmonitor/fsm-ipc-linux.c > new file mode 100644 > index 0000000000..d34a6419bc > --- /dev/null > +++ b/compat/fsmonitor/fsm-ipc-linux.c This is (almost) the exact same implementation as we have on macOS. We should probably deduplicate the logic. > diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c > new file mode 100644 > index 0000000000..04441c5120 > --- /dev/null > +++ b/compat/fsmonitor/fsm-listen-linux.c > @@ -0,0 +1,740 @@ > +#include "git-compat-util.h" > +#include "dir.h" > +#include "fsmonitor-ll.h" > +#include "fsm-listen.h" > +#include "fsmonitor--daemon.h" > +#include "fsmonitor-path-utils.h" > +#include "gettext.h" > +#include "simple-ipc.h" > +#include "string-list.h" > +#include "trace.h" > + > +#include <dirent.h> > +#include <fcntl.h> > +#include <poll.h> > +#include <sys/inotify.h> > +#include <sys/stat.h> From these we should really only require <sys/inotify.h>, as all others are included via "compat/posix.h" > +/* > + * Register an inotify watch, add watch descriptor to path mapping > + * and the reverse mapping. > + */ > +static int add_watch(const char *path, struct fsm_listen_data *data) > +{ > + const char *interned = strintern(path); > + struct watch_entry *w1, *w2; > + > + /* add the inotify watch, don't allow watches to be modified */ > + int wd = inotify_add_watch(data->fd_inotify, interned, > + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) > + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); > + if (wd < 0) { > + if (errno == ENOENT || errno == ENOTDIR) > + return 0; /* directory was deleted or is not a directory */ > + if (errno == EEXIST) > + return 0; /* watch already exists, no action needed */ Hm. In case the watch already exists, can't it be the case that we have raced e.g. during a rename? For example, the old watch hasn't been removed yet, but we already try to create the new watch. If we then see EEXIST we wouldn't update the hash map and thus keep the old path intact. Or is the order of inotify events guaranteed? > + return error_errno(_("inotify_add_watch('%s') failed"), interned); > + } > + > + /* add watch descriptor -> directory mapping */ > + CALLOC_ARRAY(w1, 1); > + w1->wd = wd; > + w1->dir = interned; > + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); > + hashmap_add(&data->watches, &w1->ent); > + > + /* add directory -> watch descriptor mapping */ > + CALLOC_ARRAY(w2, 1); > + w2->wd = wd; > + w2->dir = interned; > + hashmap_entry_init(&w2->ent, strhash(w2->dir)); > + hashmap_add(&data->revwatches, &w2->ent); We have to create two watch entries here, which is a bit puzzling at first as you'd expect that we can simply sotre the same entry twice. But this is a limitation of our hashmap interface. > + return 0; > +} > + > +/* > + * Remove the inotify watch, the watch descriptor to path mapping > + * and the reverse mapping. > + */ > +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) > +{ > + struct watch_entry k1, k2, *w1, *w2; > + > + /* remove watch, ignore error if kernel already did it */ > + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) > + error_errno(_("inotify_rm_watch() failed")); > + > + k1.wd = w->wd; > + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); > + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); > + if (!w1) > + BUG("Double remove of watch for '%s'", w->dir); Error messages should start with a lower-case letter. > +/* > + * Recursively add watches to every directory under path > + */ > +static int register_inotify(const char *path, > + struct fsmonitor_daemon_state *state, > + struct fsmonitor_batch *batch) > +{ > + DIR *dir; > + const char *rel; > + struct strbuf current = STRBUF_INIT; > + struct dirent *de; > + struct stat fs; > + int ret = -1; > + > + dir = opendir(path); > + if (!dir) { > + if (errno == ENOENT || errno == ENOTDIR) > + return 0; /* directory was deleted */ Is it correct to conflate ENOENT and ENOTDIR here? In the first case the directory was deleted, sure. But in the second case the directory might have turned into a file due to renames, so don't we have to treat it a bit differently? > + return error_errno(_("opendir('%s') failed"), path); > + } > + > + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { > + strbuf_reset(¤t); > + strbuf_addf(¤t, "%s/%s", path, de->d_name); > + if (lstat(current.buf, &fs)) { > + if (errno == ENOENT) > + continue; /* file was deleted */ > + error_errno(_("lstat('%s') failed"), current.buf); > + goto failed; > + } We don't use fstatat(3p) in our codebase yet, also because it's not easily portable to Windows. But I wonder whether we should use it here to be safer against races. > +/* > + * Process a single inotify event and queue for publication. > + */ > +static int process_event(const char *path, > + const struct inotify_event *event, > + struct fsmonitor_batch **batch, > + struct string_list *cookie_list, > + struct fsmonitor_daemon_state *state) > +{ > + const char *rel; > + const char *last_sep; > + > + switch (fsmonitor_classify_path_absolute(state, path)) { > + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: > + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: > + /* Use just the filename of the cookie file. */ > + last_sep = find_last_dir_sep(path); > + string_list_append(cookie_list, > + last_sep ? last_sep + 1 : path); > + break; > + case IS_INSIDE_DOT_GIT: > + case IS_INSIDE_GITDIR: > + break; > + case IS_DOT_GIT: > + case IS_GITDIR: > + /* > + * If .git directory is deleted or renamed away, > + * we have to quit. > + */ > + if (em_dir_deleted(event->mask)) { > + trace_printf_key(&trace_fsmonitor, > + "event: gitdir removed"); > + state->listen_data->shutdown = SHUTDOWN_FORCE; > + goto done; > + } > + > + if (em_dir_renamed(event->mask)) { > + trace_printf_key(&trace_fsmonitor, > + "event: gitdir renamed"); > + state->listen_data->shutdown = SHUTDOWN_FORCE; > + goto done; > + } > + break; > + case IS_WORKDIR_PATH: > + /* normal events in the working directory */ > + if (trace_pass_fl(&trace_fsmonitor)) > + log_mask_set(path, event->mask); > + > + if (!*batch) > + *batch = fsmonitor_batch__new(); > + > + rel = path + state->path_worktree_watch.len + 1; > + fsmonitor_batch__add_path(*batch, rel); > + > + if (em_dir_deleted(event->mask)) > + break; Curious. Don't we have to unregister the watcher in case a directory was deleted? > +/* > + * Read the inotify event stream and pre-process events before further > + * processing and eventual publishing. > + */ > +static void handle_events(struct fsmonitor_daemon_state *state) > +{ > + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ > + char buf[4096] > + __attribute__ ((aligned(__alignof__(struct inotify_event)))); > + > + struct hashmap *watches = &state->listen_data->watches; > + struct fsmonitor_batch *batch = NULL; > + struct string_list cookie_list = STRING_LIST_INIT_DUP; > + struct watch_entry k, *w; > + struct strbuf path = STRBUF_INIT; > + const struct inotify_event *event; > + int fd = state->listen_data->fd_inotify; > + ssize_t len; > + char *ptr, *p; > + > + for (;;) { > + len = read(fd, buf, sizeof(buf)); > + if (len == -1) { > + if (errno == EAGAIN || errno == EINTR) > + goto done; > + error_errno(_("reading inotify message stream failed")); > + state->listen_data->shutdown = SHUTDOWN_ERROR; > + goto done; > + } > + > + /* nothing to read */ > + if (len == 0) > + goto done; > + > + /* Loop over all events in the buffer. */ > + for (ptr = buf; ptr < buf + len; > + ptr += sizeof(struct inotify_event) + event->len) { > + > + event = (const struct inotify_event *)ptr; > + > + if (em_ignore(event->mask)) > + continue; > + > + /* File system was unmounted or event queue overflowed */ > + if (em_force_shutdown(event->mask)) { > + if (trace_pass_fl(&trace_fsmonitor)) > + log_mask_set("Forcing shutdown", event->mask); > + state->listen_data->shutdown = SHUTDOWN_FORCE; > + goto done; > + } > + > + k.wd = event->wd; > + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); > + > + w = hashmap_get_entry(watches, &k, ent, NULL); > + if (!w) { > + /* Watch was removed, skip event */ > + continue; > + } > + > + /* directory watch was removed */ > + if (em_remove_watch(event->mask)) { > + remove_watch(w, state->listen_data); > + continue; > + } Can it happen that events arrive out-of-order so that we have some events queued up that would touch the same path? In such a case we might silently ignore such queued events. > +/* > + * Non-blocking read of the inotify events stream. The inotify fd is polled > + * frequently to help minimize the number of queue overflows. > + */ > +void fsm_listen__loop(struct fsmonitor_daemon_state *state) > +{ > + int poll_num; > + const int interval = 1000; > + time_t checked = time(NULL); Do we need to use a monotonic clock here to ensure that there cannot be any backwards jumps in time, e.g. when switching from summer to winter time? > + struct pollfd fds[1]; > + > + fds[0].fd = state->listen_data->fd_inotify; > + fds[0].events = POLLIN; > + > + /* > + * Our fs event listener is now running, so it's safe to start > + * serving client requests. > + */ > + ipc_server_start_async(state->ipc_server_data); > + > + for (;;) { > + switch (state->listen_data->shutdown) { Do we have to synchronize access to `state->listen_data->shutdown`? As far as I can see it's being set by the fs event listener. > + case SHUTDOWN_CONTINUE: > + poll_num = poll(fds, 1, 1); > + if (poll_num == -1) { > + if (errno == EINTR) > + continue; > + error_errno(_("polling inotify message stream failed")); > + state->listen_data->shutdown = SHUTDOWN_ERROR; > + continue; > + } > + > + if ((time(NULL) - checked) >= interval) { Is it intended that the polling timeout is 1000 seconds, or ~16 minutes? If so, it feels like something that might warrant a comment. > diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c > new file mode 100644 > index 0000000000..fc4acbc20d > --- /dev/null > +++ b/compat/fsmonitor/fsm-path-utils-linux.c > @@ -0,0 +1,223 @@ > +#include "git-compat-util.h" > +#include "fsmonitor-ll.h" > +#include "fsmonitor-path-utils.h" > +#include "gettext.h" > +#include "trace.h" > + > +#include <errno.h> > +#include <stdio.h> > +#include <string.h> These includes shouldn't be required, as they are already included via "git-compat-util.h". [snip] > +/* > + * Find the mount point for a given path by reading /proc/mounts. > + * Returns the filesystem type for the longest matching mount point. > + */ > +static char *find_mount(const char *path, struct statfs *fs) > +{ > + FILE *fp; > + struct strbuf line = STRBUF_INIT; > + struct strbuf match = STRBUF_INIT; > + struct strbuf fstype = STRBUF_INIT; > + char *result = NULL; > + struct statfs path_fs; > + > + if (statfs(path, &path_fs) < 0) > + return NULL; > + > + fp = fopen("/proc/mounts", "r"); > + if (!fp) > + return NULL; In which cases do we need to have this fallback for statfs? This syscall exists in Linux since the 90s, so shouldn't we be able to assume that we can use it? Or are there specific error cases that we need to worry about here? > diff --git a/compat/fsmonitor/fsm-settings-linux.c b/compat/fsmonitor/fsm-settings-linux.c > new file mode 100644 > index 0000000000..23e7442d0c > --- /dev/null > +++ b/compat/fsmonitor/fsm-settings-linux.c This file is again an almost exact copy of what we have in "fsm-settings-darwin.c", and as far as I can see there isn't even anything specific to either of the systems here. So we should probably deduplicate the logic. > diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh > index 409cd0cd12..d2f1f1097e 100755 > --- a/t/t7527-builtin-fsmonitor.sh > +++ b/t/t7527-builtin-fsmonitor.sh > @@ -520,6 +520,28 @@ test_expect_success 'directory changes to a file' ' > grep "^event: dir1$" .git/trace > ' > > +test_expect_success 'rapid nested directory creation' ' > + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && > + > + start_daemon --tf "$PWD/.git/trace" && > + > + # Rapidly create nested directories to exercise race conditions > + # where directory watches may be added concurrently during > + # event processing and recursive scanning. > + for i in $(test_seq 1 20) > + do > + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 > + done && > + > + # Give the daemon time to process all events > + sleep 1 && > + > + test-tool fsmonitor-client query --token 0 && > + > + # Verify daemon is still running (did not crash) > + git fsmonitor--daemon status > +' It's a bit unclear why specifically this test was added. Does it catch an edge case that you have discovered? Might make sense to also add it in a preparatory commit so that we can get a bit of context. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v6 00/10] fsmonitor: implement filesystem change listener for Linux 2026-02-24 1:31 ` [PATCH v5] " Paul Tarjan via GitGitGadget 2026-02-24 8:03 ` Patrick Steinhardt @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget ` (10 more replies) 1 sibling, 11 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan This series implements the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. It also fixes two memory leaks in the platform-independent daemon code and deduplicates the IPC and settings logic that is now shared between macOS and Linux. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. This series builds on work from https://github.com/git/git/pull/1352 by Eric DeCosta and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v5: * Split monolithic commit into 10-patch series per Patrick's review * Deduplicated fsm-ipc and fsm-settings into shared Unix implementations * Rewrote commit message with prose paragraphs, explain inotify vs fanotify, removed "Issues addressed" sections, added Based-on-patch-by trailers * Removed redundant includes already provided by compat/posix.h * Fixed error/trace message capitalization per coding guidelines * Fixed stale rename check interval from 1000 seconds to 1 second * Changed poll timeout from 1ms to 50ms to reduce idle CPU wake-ups * Replaced infinite pthread_cond_wait cookie loop with one-second pthread_cond_timedwait (prevents daemon hangs on overlay filesystems where events are never delivered) * Added pthread_cond_timedwait to Windows pthread compatibility layer * Separated test into its own commit with smoke test that skips when inotify events are not delivered (e.g., overlayfs with older kernels) * Fixed test hang on Fedora CI: stop_git() looped forever when ps was unavailable because bash in POSIX/sh mode returns exit 0 from kill with an empty process group argument. Fixed by falling back to /proc/$pid/stat for process group ID and guarding stop_git against empty pgid. * Redirect spawn_daemon() stdout/stderr to /dev/null and close inherited file descriptors to prevent the intermediate process from holding test pipe file descriptors * Call setsid() on daemon detach to prevent shells with job control from waiting on the daemon process group * Close inherited file descriptors 3-7 in the test watchdog subprocess * Added 30-second timeout to "fsmonitor--daemon stop" to prevent indefinite blocking * Added helpful error message when inotify watch limit (max_user_watches) is reached * Initialize fd_inotify to -1 and use fd >= 0 check for correct fd 0 handling * Use sysconf(_SC_OPEN_MAX) instead of hardcoded 1024 for fd close limit * Check setsid() return value Changes since v4: * Added Meson build support Changes since v3: * Fix crash on rapid nested directory creation (EEXIST from inotify_add_watch with IN_MASK_CREATE) * Extensive stress testing Changes since v2: * Fix khash memory leak in do_handle_client Changes since v1: * Fix hashmap memory leak in fsmonitor_run_daemon() Paul Tarjan (10): fsmonitor: fix khash memory leak in do_handle_client fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon compat/win32: add pthread_cond_timedwait fsmonitor: use pthread_cond_timedwait for cookie wait fsmonitor: deduplicate IPC path logic for Unix platforms fsmonitor: deduplicate settings logic for Unix platforms fsmonitor: implement filesystem change listener for Linux fsmonitor: add tests for Linux run-command: add close_fd_above_stderr option fsmonitor: close inherited file descriptors and detach in daemon Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- Makefile | 12 +- builtin/fsmonitor--daemon.c | 71 +- compat/fsmonitor/fsm-health-linux.c | 33 + .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 4 +- compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 220 ++++++ ...-settings-darwin.c => fsm-settings-unix.c} | 24 +- compat/win32/pthread.c | 26 + compat/win32/pthread.h | 2 + config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 14 +- fsmonitor-ipc.c | 3 + meson.build | 17 +- run-command.c | 11 + run-command.h | 9 + t/t7527-builtin-fsmonitor.sh | 101 ++- 18 files changed, 1286 insertions(+), 49 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (96%) create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (82%) base-commit: 3e0db84c88c57e70ac8be8c196dfa92c5d656fbc Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v6 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v6 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v5: -: ---------- > 1: 4d4dec8fa1 fsmonitor: fix khash memory leak in do_handle_client -: ---------- > 2: d0bd3e32ca fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon -: ---------- > 3: d2c5ca0939 compat/win32: add pthread_cond_timedwait -: ---------- > 4: 0051a19303 fsmonitor: use pthread_cond_timedwait for cookie wait -: ---------- > 5: ff31e359a7 fsmonitor: deduplicate IPC path logic for Unix platforms -: ---------- > 6: 39da1e6be3 fsmonitor: deduplicate settings logic for Unix platforms 1: c54814eb31 ! 7: 4eadc06004 fsmonitor: implement filesystem change listener for Linux @@ Metadata ## Commit message ## fsmonitor: implement filesystem change listener for Linux - Implement fsmonitor for Linux using the inotify API, bringing it to - feature parity with existing Windows and macOS implementations. + Implement the built-in fsmonitor daemon for Linux using the inotify + API, bringing it to feature parity with the existing Windows and macOS + implementations. - The Linux implementation uses inotify to monitor filesystem events. - Unlike macOS's FSEvents which can watch a single root directory, - inotify requires registering watches on every directory of interest. - The implementation carefully handles directory renames and moves - using inotify's cookie mechanism to track IN_MOVED_FROM/IN_MOVED_TO - event pairs. + The implementation uses inotify rather than fanotify because fanotify + requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it + unsuitable for an unprivileged user-space daemon. While inotify has + the limitation of requiring a separate watch on every directory (unlike + macOS's FSEvents, which can monitor an entire directory tree with a + single watch), it operates without elevated privileges and provides + the per-file event granularity needed for fsmonitor. - Key implementation details: - - Uses inotify_init1(O_NONBLOCK) for non-blocking event monitoring - - Maintains bidirectional hashmaps between watch descriptors and paths - for efficient event processing - - Handles directory creation, deletion, and renames dynamically - - Detects remote filesystems (NFS, CIFS, SMB, etc.) via statfs() - - Falls back to $HOME/.git-fsmonitor-* for socket when .git is remote + The listener uses inotify_init1(O_NONBLOCK) with a poll loop that + checks for events with a 50-millisecond timeout, keeping the inotify + queue well-drained to minimize the risk of overflows. Bidirectional + hashmaps map between watch descriptors and directory paths for efficient + event resolution. Directory renames are tracked using inotify's cookie + mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a + periodic check detects stale renames where the matching IN_MOVED_TO + never arrived, forcing a resync. - Build configuration: - - Enabled via FSMONITOR_DAEMON_BACKEND=linux and FSMONITOR_OS_SETTINGS=linux - - Requires NO_PTHREADS and NO_UNIX_SOCKETS to be unset - - Adds HAVE_LINUX_MAGIC_H for filesystem type detection + New directory creation triggers recursive watch registration to ensure + all subdirectories are monitored. The IN_MASK_CREATE flag is used + where available to prevent modifying existing watches, with a fallback + for older kernels. When IN_MASK_CREATE is available and + inotify_add_watch returns EEXIST, it means another thread or recursive + scan has already registered the watch, so it is safe to ignore. - Documentation updated to note that fsmonitor.socketDir is now supported - on both Mac OS and Linux, and adds a section about inotify watch limits. - - Issues addressed from PR #1352 (git/git) review comments: - - GPLv3 ME_REMOTE macro: Rewrote remote filesystem detection from - scratch using statfs() and linux/magic.h constants (no GPLv3 code) - - Memory leak on inotify_init1 failure: Added FREE_AND_NULL cleanup - - Unsafe hashmap iteration in dtor: Collect entries first, then modify - - Missing null checks in stop_async: Added proper guard conditions - - dirname() modifying argument: Create copy with xstrdup() first - - Non-portable f_fsid.__val: Use memcmp() for fsid comparison - - Missing worktree null check: Added BUG() for null worktree - - Header updates: Use git-compat-util.h, hash_to_hex_algop() - - Code style: Use xstrdup() not xmemdupz(), proper pointer style - - Issues addressed from PR #1667 (git/git) review comments: - - EINTR handling: read() now handles both EAGAIN and EINTR - - Trailing pipe in log_mask_set: Added strbuf_strip_suffix() - - Unchecked add_watch return: Now logs failure in rename_dir() - - String building: Consolidated strbuf operations with strbuf_addf() - - Translation markers: Added _() to all error_errno() messages - - Based on work from https://github.com/git/git/pull/1352 by Eric DeCosta, - and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to - work with the current codebase and address all review feedback. + Remote filesystem detection uses statfs() to identify network-mounted + filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. + Mount point information is read from /proc/mounts and matched against + the statfs f_fsid to get accurate, human-readable filesystem type names + for logging. When the .git directory is on a remote filesystem, the + IPC socket falls back to $HOME or a user-configured directory via the + fsmonitor.socketDir setting. + Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> + Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> ## Documentation/config/fsmonitor--daemon.adoc ## @@ Documentation/git-fsmonitor--daemon.adoc: By default, the socket is created in t ------------- - ## builtin/fsmonitor--daemon.c ## -@@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state, - const struct fsmonitor_batch *batch; - struct fsmonitor_batch *remainder = NULL; - intmax_t count = 0, duplicates = 0; -- kh_str_t *shown; -+ kh_str_t *shown = NULL; - int hash_ret; - int do_trivial = 0; - int do_flush = 0; -@@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state, - total_response_len += payload.len; - } - -- kh_release_str(shown); -- - pthread_mutex_lock(&state->main_lock); - - if (token_data->client_ref_count > 0) -@@ builtin/fsmonitor--daemon.c: static int do_handle_client(struct fsmonitor_daemon_state *state, - trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); - - cleanup: -+ kh_destroy_str(shown); - strbuf_release(&response_token); - strbuf_release(&requested_token_id); - strbuf_release(&payload); -@@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void) - done: - pthread_cond_destroy(&state.cookies_cond); - pthread_mutex_destroy(&state.main_lock); -+ hashmap_clear(&state.cookies); - fsm_listen__dtor(&state); - fsm_health__dtor(&state); - - ## compat/fsmonitor/fsm-health-linux.c (new) ## @@ +#include "git-compat-util.h" @@ compat/fsmonitor/fsm-health-linux.c (new) + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ -+} - - ## compat/fsmonitor/fsm-ipc-linux.c (new) ## -@@ -+#define USE_THE_REPOSITORY_VARIABLE -+ -+#include "git-compat-util.h" -+#include "config.h" -+#include "gettext.h" -+#include "hex.h" -+#include "path.h" -+#include "repository.h" -+#include "strbuf.h" -+#include "fsmonitor-ll.h" -+#include "fsmonitor-ipc.h" -+#include "fsmonitor-path-utils.h" -+ -+static GIT_PATH_FUNC(fsmonitor_ipc__get_default_path, "fsmonitor--daemon.ipc") -+ -+const char *fsmonitor_ipc__get_path(struct repository *r) -+{ -+ static const char *ipc_path = NULL; -+ git_SHA_CTX sha1ctx; -+ char *sock_dir = NULL; -+ struct strbuf ipc_file = STRBUF_INIT; -+ unsigned char hash[GIT_SHA1_RAWSZ]; -+ -+ if (!r) -+ BUG("No repository passed into fsmonitor_ipc__get_path"); -+ -+ if (ipc_path) -+ return ipc_path; -+ -+ /* By default the socket file is created in the .git directory */ -+ if (fsmonitor__is_fs_remote(r->gitdir) < 1) { -+ ipc_path = fsmonitor_ipc__get_default_path(); -+ return ipc_path; -+ } -+ -+ if (!r->worktree) -+ BUG("repository has no worktree"); -+ -+ git_SHA1_Init(&sha1ctx); -+ git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); -+ git_SHA1_Final(hash, &sha1ctx); -+ -+ repo_config_get_string(r, "fsmonitor.socketdir", &sock_dir); -+ -+ /* Create the socket file in either socketDir or $HOME */ -+ if (sock_dir && *sock_dir) { -+ strbuf_addf(&ipc_file, "%s/.git-fsmonitor-%s", -+ sock_dir, hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); -+ } else { -+ strbuf_addf(&ipc_file, "~/.git-fsmonitor-%s", -+ hash_to_hex_algop(hash, &hash_algos[GIT_HASH_SHA1])); -+ } -+ free(sock_dir); -+ -+ ipc_path = interpolate_path(ipc_file.buf, 1); -+ if (!ipc_path) -+ die(_("Invalid path: %s"), ipc_file.buf); -+ -+ strbuf_release(&ipc_file); -+ return ipc_path; +} ## compat/fsmonitor/fsm-listen-linux.c (new) ## @@ compat/fsmonitor/fsm-listen-linux.c (new) +#include "string-list.h" +#include "trace.h" + -+#include <dirent.h> -+#include <fcntl.h> -+#include <poll.h> +#include <sys/inotify.h> -+#include <sys/stat.h> + +/* + * Safe value to bitwise OR with rest of mask for @@ compat/fsmonitor/fsm-listen-linux.c (new) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ ++ if (errno == ENOSPC) ++ return error(_("inotify watch limit reached; " ++ "increase fs.inotify.max_user_watches")); + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + @@ compat/fsmonitor/fsm-listen-linux.c (new) + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) -+ BUG("Double remove of watch for '%s'", w->dir); ++ BUG("double remove of watch for '%s'", w->dir); + + if (w1->cookie) -+ BUG("Removing watch for '%s' which has a pending rename", w1->dir); ++ BUG("removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) -+ BUG("Double remove of reverse watch for '%s'", w->dir); ++ BUG("double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); @@ compat/fsmonitor/fsm-listen-linux.c (new) + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, -+ "No matching watch for rename to '%s'", path); ++ "no matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, -+ "No matching cookie for rename to '%s'", path); ++ "no matching cookie for rename to '%s'", path); + } +} + @@ compat/fsmonitor/fsm-listen-linux.c (new) + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; ++ data->fd_inotify = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); @@ compat/fsmonitor/fsm-listen-linux.c (new) + + FREE_AND_NULL(state->listen_data); + -+ if (fd && (close(fd) < 0)) ++ if (fd >= 0 && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + @@ compat/fsmonitor/fsm-listen-linux.c (new) + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); -+ state->listen_data->shutdown = SHUTDOWN_ERROR; ++ state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + @@ compat/fsmonitor/fsm-listen-linux.c (new) +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; -+ const int interval = 1000; ++ /* ++ * Interval in seconds between checks for stale directory renames. ++ * A directory rename that is not completed within this window ++ * (i.e. no matching IN_MOVED_TO for an IN_MOVED_FROM) indicates ++ * missed events, forcing a shutdown. ++ */ ++ const int interval = 1; + time_t checked = time(NULL); + struct pollfd fds[1]; + @@ compat/fsmonitor/fsm-listen-linux.c (new) + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: -+ poll_num = poll(fds, 1, 1); ++ poll_num = poll(fds, 1, 50); + if (poll_num == -1) { + if (errno == EINTR) + continue; @@ compat/fsmonitor/fsm-listen-linux.c (new) + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, -+ "Missed IN_MOVED_TO events, forcing shutdown"); ++ "missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } @@ compat/fsmonitor/fsm-path-utils-linux.c (new) +#include "gettext.h" +#include "trace.h" + -+#include <errno.h> -+#include <stdio.h> -+#include <string.h> +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H @@ compat/fsmonitor/fsm-path-utils-linux.c (new) + const struct alias_info *info UNUSED) +{ + return NULL; -+} - - ## compat/fsmonitor/fsm-settings-linux.c (new) ## -@@ -+#include "git-compat-util.h" -+#include "config.h" -+#include "fsmonitor-ll.h" -+#include "fsmonitor-ipc.h" -+#include "fsmonitor-settings.h" -+#include "fsmonitor-path-utils.h" -+ -+#include <libgen.h> -+ -+/* -+ * For the builtin FSMonitor, we create the Unix domain socket for the -+ * IPC in the .git directory. If the working directory is remote, -+ * then the socket will be created on the remote file system. This -+ * can fail if the remote file system does not support UDS file types -+ * (e.g. smbfs to a Windows server) or if the remote kernel does not -+ * allow a non-local process to bind() the socket. (These problems -+ * could be fixed by moving the UDS out of the .git directory and to a -+ * well-known local directory on the client machine, but care should -+ * be taken to ensure that $HOME is actually local and not a managed -+ * file share.) -+ * -+ * FAT32 and NTFS working directories are problematic too. -+ * -+ * The builtin FSMonitor uses a Unix domain socket in the .git -+ * directory for IPC. These Windows drive formats do not support -+ * Unix domain sockets, so mark them as incompatible for the daemon. -+ */ -+static enum fsmonitor_reason check_uds_volume(struct repository *r) -+{ -+ struct fs_info fs; -+ const char *ipc_path = fsmonitor_ipc__get_path(r); -+ char *path; -+ char *dir; -+ -+ /* -+ * Create a copy for dirname() since it may modify its argument. -+ */ -+ path = xstrdup(ipc_path); -+ dir = dirname(path); -+ -+ if (fsmonitor__get_fs_info(dir, &fs) == -1) { -+ free(path); -+ return FSMONITOR_REASON_ERROR; -+ } -+ -+ free(path); -+ -+ if (fs.is_remote || -+ !strcmp(fs.typename, "msdos") || -+ !strcmp(fs.typename, "ntfs") || -+ !strcmp(fs.typename, "vfat")) { -+ free(fs.typename); -+ return FSMONITOR_REASON_NOSOCKETS; -+ } -+ -+ free(fs.typename); -+ return FSMONITOR_REASON_OK; -+} -+ -+enum fsmonitor_reason fsm_os__incompatible(struct repository *r, int ipc) -+{ -+ enum fsmonitor_reason reason; -+ -+ if (ipc) { -+ reason = check_uds_volume(r); -+ if (reason != FSMONITOR_REASON_OK) -+ return reason; -+ } -+ -+ return FSMONITOR_REASON_OK; +} ## config.mak.uname ## @@ config.mak.uname: ifeq ($(uname_S),Linux) ## contrib/buildsystems/CMakeLists.txt ## @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) - add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) ++ add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + add_compile_definitions(HAVE_LINUX_MAGIC_H) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-linux.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c) + -+ add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-linux.c) + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) endif() - endif() - ## meson.build ## @@ meson.build: endif @@ meson.build: endif elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' libgit_dependencies += dependency('CoreServices') - - ## t/t7527-builtin-fsmonitor.sh ## -@@ t/t7527-builtin-fsmonitor.sh: test_expect_success 'directory changes to a file' ' - grep "^event: dir1$" .git/trace - ' - -+test_expect_success 'rapid nested directory creation' ' -+ test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && -+ -+ start_daemon --tf "$PWD/.git/trace" && -+ -+ # Rapidly create nested directories to exercise race conditions -+ # where directory watches may be added concurrently during -+ # event processing and recursive scanning. -+ for i in $(test_seq 1 20) -+ do -+ mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 -+ done && -+ -+ # Give the daemon time to process all events -+ sleep 1 && -+ -+ test-tool fsmonitor-client query --token 0 && -+ -+ # Verify daemon is still running (did not crash) -+ git fsmonitor--daemon status -+' -+ - # The next few test cases exercise the token-resync code. When filesystem - # drops events (because of filesystem velocity or because the daemon isn't - # polling fast enough), we need to discard the cached data (relative to the -: ---------- > 8: 8fec92d5b4 fsmonitor: add tests for Linux -: ---------- > 9: 817489b3ea run-command: add close_fd_above_stderr option -: ---------- > 10: bb438afbbe fsmonitor: close inherited file descriptors and detach in daemon -- gitgitgadget ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v6 01/10] fsmonitor: fix khash memory leak in do_handle_client 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 21:01 ` Junio C Hamano 2026-02-25 20:17 ` [PATCH v6 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget ` (9 subsequent siblings) 10 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `shown` kh_str_t was freed with kh_release_str() at a point in the code only reachable in the non-trivial response path. When the client receives a trivial response, the code jumps to the `cleanup` label, skipping the kh_release_str() call entirely and leaking the hash table. Fix this by initializing `shown` to NULL and moving the cleanup to the `cleanup` label using kh_destroy_str(), which is safe to call on NULL. This ensures the hash table is freed regardless of which code path is taken. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..bc4571938c 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v6 01/10] fsmonitor: fix khash memory leak in do_handle_client 2026-02-25 20:17 ` [PATCH v6 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-02-25 21:01 ` Junio C Hamano 0 siblings, 0 replies; 129+ messages in thread From: Junio C Hamano @ 2026-02-25 21:01 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Patrick Steinhardt, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Paul Tarjan <github@paulisageek.com> > > The `shown` kh_str_t was freed with kh_release_str() at a point in > the code only reachable in the non-trivial response path. When the > client receives a trivial response, the code jumps to the `cleanup` > label, skipping the kh_release_str() call entirely and leaking the > hash table. > > Fix this by initializing `shown` to NULL and moving the cleanup to the > `cleanup` label using kh_destroy_str(), which is safe to call on NULL. > This ensures the hash table is freed regardless of which code path is > taken. Makes perfect sense, the changes to the code matches the description, and the difference between kh_release_* and kh_destroy_* in khash.h is exactly as described in the log message. Perfect. I wish all the patches posted here are as easy to review as this one ;-) > Signed-off-by: Paul Tarjan <github@paulisageek.com> > --- > builtin/fsmonitor--daemon.c | 5 ++--- > 1 file changed, 2 insertions(+), 3 deletions(-) > > diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c > index 242c594646..bc4571938c 100644 > --- a/builtin/fsmonitor--daemon.c > +++ b/builtin/fsmonitor--daemon.c > @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > const struct fsmonitor_batch *batch; > struct fsmonitor_batch *remainder = NULL; > intmax_t count = 0, duplicates = 0; > - kh_str_t *shown; > + kh_str_t *shown = NULL; > int hash_ret; > int do_trivial = 0; > int do_flush = 0; > @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > total_response_len += payload.len; > } > > - kh_release_str(shown); > - > pthread_mutex_lock(&state->main_lock); > > if (token_data->client_ref_count > 0) > @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); > > cleanup: > + kh_destroy_str(shown); > strbuf_release(&response_token); > strbuf_release(&requested_token_id); > strbuf_release(&payload); ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v6 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 03/10] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget ` (8 subsequent siblings) 10 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `state.cookies` hashmap is initialized during daemon startup but never freed during cleanup in the `done:` label of fsmonitor_run_daemon(). Add a hashmap_clear() call to prevent this memory leak. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 1 + 1 file changed, 1 insertion(+) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index bc4571938c..4d52622e24 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1404,6 +1404,7 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + hashmap_clear(&state.cookies); fsm_listen__dtor(&state); fsm_health__dtor(&state); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v6 03/10] compat/win32: add pthread_cond_timedwait 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget ` (7 subsequent siblings) 10 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pthread_cond_timedwait() implementation to the Windows pthread compatibility layer using SleepConditionVariableCS() with a millisecond timeout computed from the absolute deadline. This enables callers to use bounded waits on condition variables instead of blocking indefinitely with pthread_cond_wait(). Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/win32/pthread.c | 26 ++++++++++++++++++++++++++ compat/win32/pthread.h | 2 ++ 2 files changed, 28 insertions(+) diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c index 7e93146963..538ef92d9d 100644 --- a/compat/win32/pthread.c +++ b/compat/win32/pthread.c @@ -66,3 +66,29 @@ int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) return err_win_to_posix(GetLastError()); return 0; } + +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime) +{ + struct timeval now; + long long now_ms, deadline_ms; + DWORD timeout_ms; + + gettimeofday(&now, NULL); + now_ms = (long long)now.tv_sec * 1000 + now.tv_usec / 1000; + deadline_ms = (long long)abstime->tv_sec * 1000 + + abstime->tv_nsec / 1000000; + + if (deadline_ms <= now_ms) + timeout_ms = 0; + else + timeout_ms = (DWORD)(deadline_ms - now_ms); + + if (SleepConditionVariableCS(cond, mutex, timeout_ms) == 0) { + DWORD err = GetLastError(); + if (err == ERROR_TIMEOUT) + return ETIMEDOUT; + return err_win_to_posix(err); + } + return 0; +} diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index ccacc5a53b..d80df8d12a 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -64,6 +64,8 @@ int win32_pthread_join(pthread_t *thread, void **value_ptr); pthread_t pthread_self(void); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime); static inline void NORETURN pthread_exit(void *ret) { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (2 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 03/10] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 21:13 ` Junio C Hamano 2026-02-25 21:17 ` Junio C Hamano 2026-02-25 20:17 ` [PATCH v6 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget ` (6 subsequent siblings) 10 siblings, 2 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The cookie wait in with_lock__wait_for_cookie() uses an infinite pthread_cond_wait() loop. The existing comment notes the desire to switch to pthread_cond_timedwait(), but the routine was not available in git thread-utils. On certain container or overlay filesystems, inotify watches may succeed but events are never delivered. In this case the daemon would hang indefinitely waiting for the cookie event, which in turn causes the client to hang. Replace the infinite wait with a one-second timeout using pthread_cond_timedwait(). If the timeout fires, report an error and let the client proceed with a trivial (full-scan) response rather than blocking forever. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 4d52622e24..f6c406ff12 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -197,20 +197,31 @@ static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie( unlink(cookie_pathname.buf); /* - * Technically, this is an infinite wait (well, unless another - * thread sends us an abort). I'd like to change this to - * use `pthread_cond_timedwait()` and return an error/timeout - * and let the caller do the trivial response thing, but we - * don't have that routine in our thread-utils. - * - * After extensive beta testing I'm not really worried about - * this. Also note that the above open() and unlink() calls - * will cause at least two FS events on that path, so the odds - * of getting stuck are pretty slim. + * Wait for the listener thread to see the cookie file. + * Time out after a short interval so that the client + * does not hang forever if the filesystem does not deliver + * events (e.g., on certain container/overlay filesystems + * where inotify watches succeed but events never arrive). */ - while (cookie->result == FCIR_INIT) - pthread_cond_wait(&state->cookies_cond, - &state->main_lock); + { + struct timeval now; + struct timespec ts; + int err = 0; + + gettimeofday(&now, NULL); + ts.tv_sec = now.tv_sec + 1; + ts.tv_nsec = now.tv_usec * 1000; + + while (cookie->result == FCIR_INIT && !err) + err = pthread_cond_timedwait(&state->cookies_cond, + &state->main_lock, + &ts); + if (err == ETIMEDOUT && cookie->result == FCIR_INIT) { + trace_printf_key(&trace_fsmonitor, + "cookie_wait timed out"); + cookie->result = FCIR_ERROR; + } + } done: hashmap_remove(&state->cookies, &cookie->entry, NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-25 20:17 ` [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-02-25 21:13 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 2026-02-25 21:17 ` Junio C Hamano 1 sibling, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-25 21:13 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Patrick Steinhardt, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Paul Tarjan <github@paulisageek.com> > > The cookie wait in with_lock__wait_for_cookie() uses an infinite > pthread_cond_wait() loop. The existing comment notes the desire > to switch to pthread_cond_timedwait(), but the routine was not > available in git thread-utils. > > On certain container or overlay filesystems, inotify watches may > succeed but events are never delivered. In this case the daemon > would hang indefinitely waiting for the cookie event, which in > turn causes the client to hang. > > Replace the infinite wait with a one-second timeout using > pthread_cond_timedwait(). If the timeout fires, report an > error and let the client proceed with a trivial (full-scan) > response rather than blocking forever. I cannot convince myself if one-second interval is not too frequent to force everybody, including those with working inotify, to poll. I wonder if this is something that may want to be configurable (or better yet, auto-detectable, but that may be wishing for moon). > Signed-off-by: Paul Tarjan <github@paulisageek.com> > --- > builtin/fsmonitor--daemon.c | 37 ++++++++++++++++++++++++------------- > 1 file changed, 24 insertions(+), 13 deletions(-) > > diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c > index 4d52622e24..f6c406ff12 100644 > --- a/builtin/fsmonitor--daemon.c > +++ b/builtin/fsmonitor--daemon.c > @@ -197,20 +197,31 @@ static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie( > unlink(cookie_pathname.buf); > > /* > - * Technically, this is an infinite wait (well, unless another > - * thread sends us an abort). I'd like to change this to > - * use `pthread_cond_timedwait()` and return an error/timeout > - * and let the caller do the trivial response thing, but we > - * don't have that routine in our thread-utils. > - * > - * After extensive beta testing I'm not really worried about > - * this. Also note that the above open() and unlink() calls > - * will cause at least two FS events on that path, so the odds > - * of getting stuck are pretty slim. > + * Wait for the listener thread to see the cookie file. > + * Time out after a short interval so that the client > + * does not hang forever if the filesystem does not deliver > + * events (e.g., on certain container/overlay filesystems > + * where inotify watches succeed but events never arrive). > */ > - while (cookie->result == FCIR_INIT) > - pthread_cond_wait(&state->cookies_cond, > - &state->main_lock); > + { > + struct timeval now; > + struct timespec ts; > + int err = 0; > + > + gettimeofday(&now, NULL); > + ts.tv_sec = now.tv_sec + 1; > + ts.tv_nsec = now.tv_usec * 1000; > + > + while (cookie->result == FCIR_INIT && !err) > + err = pthread_cond_timedwait(&state->cookies_cond, > + &state->main_lock, > + &ts); > + if (err == ETIMEDOUT && cookie->result == FCIR_INIT) { > + trace_printf_key(&trace_fsmonitor, > + "cookie_wait timed out"); > + cookie->result = FCIR_ERROR; > + } > + } > > done: > hashmap_remove(&state->cookies, &cookie->entry, NULL); ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-25 21:13 ` Junio C Hamano @ 2026-02-27 6:31 ` Paul Tarjan 2026-02-27 16:44 ` Junio C Hamano 0 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan @ 2026-02-27 6:31 UTC (permalink / raw) To: git; +Cc: gitster, Paul Tarjan Junio C Hamano <gitster@pobox.com> writes: > I cannot convince myself if one-second interval is not too frequent > to force everybody, including those with working inotify, to poll. > I wonder if this is something that may want to be configurable (or > better yet, auto-detectable, but that may be wishing for moon). The 1-second timeout only fires when the filesystem fails to deliver the cookie event at all (e.g. overlayfs in containers where inotify watches succeed but events never arrive). On a working filesystem the cookie event comes back in well under a millisecond, so the timeout never triggers. When it does fire, the client falls back to a full scan, which is the safe default. Happy to make it configurable if you think that's worth it, but the current behavior seemed reasonable as a starting point. Thanks, Paul ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-27 6:31 ` Paul Tarjan @ 2026-02-27 16:44 ` Junio C Hamano 2026-02-28 0:28 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-27 16:44 UTC (permalink / raw) To: Paul Tarjan; +Cc: git, Paul Tarjan Paul Tarjan <paul@paultarjan.com> writes: > The 1-second timeout only fires when the filesystem fails to deliver > the cookie event at all (e.g. overlayfs in containers where inotify > watches succeed but events never arrive). On a working filesystem > the cookie event comes back in well under a millisecond, so the > timeout never triggers. I am not worried about that case. When the filesystem is quiescent and there is absolutely nothing fsmonitor needs to report, wouldn't we see no "cookie event" delivered at all? > When it does fire, the client falls back to > a full scan, which is the safe default. And if a every-one-second timeout forces somebody to fall back to a full scan every second, that does not sound like a safe default to me. I am clearly missing something here. Are we handling two different kind of events, one that wakes us up to expect "cookie" events, and the other "cookie" events, and we know the delivery of the former is reliable while the latter not? So on a quiescent filesystem we do not even get the first kind of event to wake us up, and we do not start waiting for "cookie" events with 1-sec timeout in the first place? If so, that does sound like a good arrangement. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-27 16:44 ` Junio C Hamano @ 2026-02-28 0:28 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-02-28 0:28 UTC (permalink / raw) To: git; +Cc: gitster, Paul Tarjan Junio C Hamano <gitster@pobox.com> writes: > I am clearly missing something here. Are we handling two different > kind of events, one that wakes us up to expect "cookie" events, and > the other "cookie" events, and we know the delivery of the former is > reliable while the latter not? So on a quiescent filesystem we do > not even get the first kind of event to wake us up, and we do not > start waiting for "cookie" events with 1-sec timeout in the first > place? If so, that does sound like a good arrangement. Yes, that's exactly right. The cookie wait only runs when a client connects and asks for the current status. The daemon creates a temporary cookie file, then waits for the listener thread to see the inotify event for that file. On a quiescent filesystem with no clients asking, this code path never executes and the timeout never fires. So the only time the 1-second timeout can trigger is when a client is actively waiting for a response and the filesystem isn't delivering events at all. In that case falling back to a full scan is the right thing to do since we can't trust the event stream anyway. Thanks, Paul ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-25 20:17 ` [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget 2026-02-25 21:13 ` Junio C Hamano @ 2026-02-25 21:17 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 1 sibling, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-25 21:17 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Patrick Steinhardt, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > + * Wait for the listener thread to see the cookie file. Is this a complete sentence? Or perhaps something like "see" -> "check"? ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-25 21:17 ` Junio C Hamano @ 2026-02-27 6:31 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-02-27 6:31 UTC (permalink / raw) To: git; +Cc: gitster, Paul Tarjan Junio C Hamano <gitster@pobox.com> writes: > Is this a complete sentence? Or perhaps something like "see" -> > "check"? Fixed, changed to "observe". Thanks, Paul ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v6 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (3 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 21:30 ` Junio C Hamano 2026-02-25 20:17 ` [PATCH v6 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget ` (5 subsequent siblings) 10 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The IPC path logic for determining the Unix domain socket location is nearly identical between macOS and Linux. Both need to check whether the .git directory is on a remote filesystem and, if so, fall back to a socket path under $HOME or a user-configured directory. Merge the two implementations into a single fsm-ipc-unix.c that is shared by both platforms. The unified version includes the worktree NULL check (BUG guard) from the Linux implementation, which was missing in the macOS version. Update Makefile, meson.build, and CMakeLists.txt to use the new shared file for non-Windows platforms. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 6 +++++- compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 4 +++- contrib/buildsystems/CMakeLists.txt | 2 +- meson.build | 7 ++++++- 4 files changed, 15 insertions(+), 4 deletions(-) rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (96%) diff --git a/Makefile b/Makefile index 8aa489f3b6..2157bbf173 100644 --- a/Makefile +++ b/Makefile @@ -2365,7 +2365,11 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o +ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o +else + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o +endif endif ifdef FSMONITOR_OS_SETTINGS diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-unix.c similarity index 96% rename from compat/fsmonitor/fsm-ipc-darwin.c rename to compat/fsmonitor/fsm-ipc-unix.c index fe149a1b37..d34a6419bc 100644 --- a/compat/fsmonitor/fsm-ipc-darwin.c +++ b/compat/fsmonitor/fsm-ipc-unix.c @@ -27,13 +27,15 @@ const char *fsmonitor_ipc__get_path(struct repository *r) if (ipc_path) return ipc_path; - /* By default the socket file is created in the .git directory */ if (fsmonitor__is_fs_remote(r->gitdir) < 1) { ipc_path = fsmonitor_ipc__get_default_path(); return ipc_path; } + if (!r->worktree) + BUG("repository has no worktree"); + git_SHA1_Init(&sha1ctx); git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); git_SHA1_Final(hash, &sha1ctx); diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..32ef6ebe1b 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -303,7 +303,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) diff --git a/meson.build b/meson.build index dd52efd1c8..8de795f9d4 100644 --- a/meson.build +++ b/meson.build @@ -1332,11 +1332,16 @@ if fsmonitor_backend != '' libgit_sources += [ 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] + + if fsmonitor_backend == 'win32' + libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' + else + libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' + endif endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v6 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms 2026-02-25 20:17 ` [PATCH v6 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget @ 2026-02-25 21:30 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-25 21:30 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Patrick Steinhardt, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Paul Tarjan <github@paulisageek.com> > > The IPC path logic for determining the Unix domain socket location is > nearly identical between macOS and Linux. Both need to check whether > the .git directory is on a remote filesystem and, if so, fall back to > a socket path under $HOME or a user-configured directory. > > Merge the two implementations into a single fsm-ipc-unix.c that is > shared by both platforms. The unified version includes the worktree > NULL check (BUG guard) from the Linux implementation, which was missing > in the macOS version. > > Update Makefile, meson.build, and CMakeLists.txt to use the new shared > file for non-Windows platforms. This sounds as if the patch started with two IPC path logic for macOS and Linux, and the patch removes one of them and updates the other one so that both platforms can use the surviving one. But the patch seems to indicate somewhat different story. The code before this patch started with a single macOS (darwin) one, but because it is mostly applicable to other UNIX variants as well, the patch renames the existing macOS one for unix and makes a small adjustment (namely, asserts that r->worktree is not NULL). Perhaps you started with two separate implementations (possibly with a new one called UNIX that was added by largely copying and pasting from the macOS one) and unified them, but that history does not exist in this 10-patch series, so the above story would need to be rewritten to say what actually is happening in this series, e.g., we realized macOS one is applicable generally for UNIX variants so we are renaming it from darwin to unix, or something. > @@ -2365,7 +2365,11 @@ ifdef FSMONITOR_DAEMON_BACKEND > COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND > COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o > COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o > - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o > +ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) > + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o > +else > + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o > +endif Makes me wonder if doing #define FSMONITOR_DAEMON_BACKEND unix for macOS and then keeping COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o as-is would be cleaner. Later, we can add the same "#define FSMONITOR_DAEMON_BACKEND unix" for Linux, perhaps? This is doubly true when we look at the build recipe changes to the meson one below ... > diff --git a/meson.build b/meson.build > index dd52efd1c8..8de795f9d4 100644 > --- a/meson.build > +++ b/meson.build > @@ -1332,11 +1332,16 @@ if fsmonitor_backend != '' > > libgit_sources += [ > 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', > - 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', > 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', > 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', > 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', > ] > + > + if fsmonitor_backend == 'win32' > + libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' > + else > + libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' > + endif > endif ... which would not be needed if fsmonitor_backend is set to 'unix' instead of 'darwin'. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v6 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms 2026-02-25 21:30 ` Junio C Hamano @ 2026-02-27 6:31 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-02-27 6:31 UTC (permalink / raw) To: git; +Cc: gitster, Paul Tarjan Junio C Hamano <gitster@pobox.com> writes: > But the patch seems to indicate somewhat different story. The code > before this patch started with a single macOS (darwin) one, but > because it is mostly applicable to other UNIX variants as well, the > patch renames the existing macOS one for unix and makes a small > adjustment (namely, asserts that r->worktree is not NULL). Good point. Restructured: FSMONITOR_OS_SETTINGS is now set to "unix" for both macOS and Linux (and "win32" for Windows). The build files use $(FSMONITOR_OS_SETTINGS) for fsm-ipc and fsm-settings, and $(FSMONITOR_DAEMON_BACKEND) for the platform-specific files (listen, health, path-utils). No more if/else blocks. Also rewrote both commit messages to describe what's actually happening (renaming darwin to unix) rather than "merging two implementations". Thanks, Paul ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v6 06/10] fsmonitor: deduplicate settings logic for Unix platforms 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (4 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 21:31 ` Junio C Hamano 2026-02-25 20:17 ` [PATCH v6 07/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (4 subsequent siblings) 10 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor settings logic for checking Unix domain socket compatibility is nearly identical between macOS and Linux. Both check whether the IPC socket directory resides on a remote filesystem or a filesystem that does not support Unix domain sockets (NTFS, FAT32, etc). Merge the two implementations into a single fsm-settings-unix.c shared by both platforms. The unified version uses the safer xstrdup() + dirname() approach from the macOS implementation (avoiding strbuf mutation with dirname()) and includes the "vfat" filesystem check. Update Makefile, meson.build, and CMakeLists.txt to use the new shared file for non-Windows platforms. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 6 ++++- ...-settings-darwin.c => fsm-settings-unix.c} | 24 ++++++++++++------- contrib/buildsystems/CMakeLists.txt | 2 +- meson.build | 11 ++++++--- 4 files changed, 29 insertions(+), 14 deletions(-) rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (82%) diff --git a/Makefile b/Makefile index 2157bbf173..febdaeb42c 100644 --- a/Makefile +++ b/Makefile @@ -2374,7 +2374,11 @@ endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS - COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o +ifeq ($(FSMONITOR_OS_SETTINGS),win32) + COMPAT_OBJS += compat/fsmonitor/fsm-settings-win32.o +else + COMPAT_OBJS += compat/fsmonitor/fsm-settings-unix.o +endif COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o endif diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-unix.c similarity index 82% rename from compat/fsmonitor/fsm-settings-darwin.c rename to compat/fsmonitor/fsm-settings-unix.c index a382590635..27d89207af 100644 --- a/compat/fsmonitor/fsm-settings-darwin.c +++ b/compat/fsmonitor/fsm-settings-unix.c @@ -5,7 +5,7 @@ #include "fsmonitor-settings.h" #include "fsmonitor-path-utils.h" - /* +/* * For the builtin FSMonitor, we create the Unix domain socket for the * IPC in the .git directory. If the working directory is remote, * then the socket will be created on the remote file system. This @@ -22,25 +22,31 @@ * The builtin FSMonitor uses a Unix domain socket in the .git * directory for IPC. These Windows drive formats do not support * Unix domain sockets, so mark them as incompatible for the daemon. - * */ static enum fsmonitor_reason check_uds_volume(struct repository *r) { struct fs_info fs; const char *ipc_path = fsmonitor_ipc__get_path(r); - struct strbuf path = STRBUF_INIT; - strbuf_add(&path, ipc_path, strlen(ipc_path)); + char *path; + char *dir; + + /* + * Create a copy for dirname() since it may modify its argument. + */ + path = xstrdup(ipc_path); + dir = dirname(path); - if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) { - strbuf_release(&path); + if (fsmonitor__get_fs_info(dir, &fs) == -1) { + free(path); return FSMONITOR_REASON_ERROR; } - strbuf_release(&path); + free(path); if (fs.is_remote || - !strcmp(fs.typename, "msdos") || - !strcmp(fs.typename, "ntfs")) { + !strcmp(fs.typename, "msdos") || + !strcmp(fs.typename, "ntfs") || + !strcmp(fs.typename, "vfat")) { free(fs.typename); return FSMONITOR_REASON_NOSOCKETS; } diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 32ef6ebe1b..0eba0c2c98 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -307,7 +307,7 @@ if(SUPPORTS_SIMPLE_IPC) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) endif() endif() diff --git a/meson.build b/meson.build index 8de795f9d4..e02f9708da 100644 --- a/meson.build +++ b/meson.build @@ -1334,13 +1334,18 @@ if fsmonitor_backend != '' 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] if fsmonitor_backend == 'win32' - libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' + libgit_sources += [ + 'compat/fsmonitor/fsm-ipc-win32.c', + 'compat/fsmonitor/fsm-settings-win32.c', + ] else - libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' + libgit_sources += [ + 'compat/fsmonitor/fsm-ipc-unix.c', + 'compat/fsmonitor/fsm-settings-unix.c', + ] endif endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v6 06/10] fsmonitor: deduplicate settings logic for Unix platforms 2026-02-25 20:17 ` [PATCH v6 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget @ 2026-02-25 21:31 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-02-25 21:31 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Patrick Steinhardt, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Paul Tarjan <github@paulisageek.com> > > The fsmonitor settings logic for checking Unix domain socket > compatibility is nearly identical between macOS and Linux. Both check > whether the IPC socket directory resides on a remote filesystem or a > filesystem that does not support Unix domain sockets (NTFS, FAT32, etc). > > Merge the two implementations into a single fsm-settings-unix.c shared > by both platforms. The unified version uses the safer xstrdup() + > dirname() approach from the macOS implementation (avoiding strbuf > mutation with dirname()) and includes the "vfat" filesystem check. > > Update Makefile, meson.build, and CMakeLists.txt to use the new shared > file for non-Windows platforms. I guess exactly the same comment as the one for 05/10 applies here as well? ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v6 06/10] fsmonitor: deduplicate settings logic for Unix platforms 2026-02-25 21:31 ` Junio C Hamano @ 2026-02-27 6:31 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-02-27 6:31 UTC (permalink / raw) To: git; +Cc: gitster, Paul Tarjan Junio C Hamano <gitster@pobox.com> writes: > I guess exactly the same comment as the one for 05/10 applies here > as well? Yep, same fix applied here. Also rewrote both commit messages to describe what's actually happening (renaming darwin to unix) rather than "merging two implementations". Thanks, Paul ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v6 07/10] fsmonitor: implement filesystem change listener for Linux 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (5 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 08/10] fsmonitor: add tests " Paul Tarjan via GitGitGadget ` (3 subsequent siblings) 10 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS's FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify's cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 220 ++++++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 10 + meson.build | 3 + 8 files changed, 1048 insertions(+), 6 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..36a701c06a --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,746 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <sys/inotify.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + if (errno == ENOSPC) + return error(_("inotify watch limit reached; " + "increase fs.inotify.max_user_watches")); + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->fd_inotify = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd >= 0 && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("Forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + /* + * Interval in seconds between checks for stale directory renames. + * A directory rename that is not completed within this window + * (i.e. no matching IN_MOVED_TO for an IN_MOVED_FROM) indicates + * missed events, forcing a shutdown. + */ + const int interval = 1; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 50); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..b4c19e0655 --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,220 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef V9FS_MAGIC +#define V9FS_MAGIC 0x01021997 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Get the filesystem type name for logging purposes. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case V9FS_MAGIC: + return "9p"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * Returns the filesystem type for the longest matching mount point. + */ +static char *find_mount(const char *path, struct statfs *fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + struct statfs path_fs; + + if (statfs(path, &path_fs) < 0) + return NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + *fs = mount_fs; + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/config.mak.uname b/config.mak.uname index 3c35ae33a3..e5d79493e5 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = linux + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 0eba0c2c98..8b4387c5a1 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -306,6 +306,16 @@ if(SUPPORTS_SIMPLE_IPC) list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) + add_compile_definitions(HAVE_LINUX_MAGIC_H) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c) + add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) endif() diff --git a/meson.build b/meson.build index e02f9708da..5661df7b3d 100644 --- a/meson.build +++ b/meson.build @@ -1322,6 +1322,9 @@ endif fsmonitor_backend = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' libgit_dependencies += dependency('CoreServices') -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v6 08/10] fsmonitor: add tests for Linux 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (6 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 07/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 09/10] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget ` (2 subsequent siblings) 10 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a smoke test that verifies the filesystem actually delivers inotify events to the daemon. On some configurations (e.g., overlayfs with older kernels), inotify watches succeed but events are never delivered. The daemon cookie wait will time out, but every subsequent test would fail. Skip the entire test file early when this is detected. Add a test that exercises rapid nested directory creation to verify the daemon correctly handles the EEXIST race between recursive scan and queued inotify events. When IN_MASK_CREATE is available and a directory watch is added during recursive registration, the kernel may also deliver a queued IN_CREATE event for the same directory. The second inotify_add_watch() returns EEXIST, which must be treated as harmless. An earlier version of the listener crashed in this scenario. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- t/meson.build | 8 +++- t/t7527-builtin-fsmonitor.sh | 89 +++++++++++++++++++++++++++++++++--- 2 files changed, 89 insertions(+), 8 deletions(-) diff --git a/t/meson.build b/t/meson.build index 19e8306298..85ef2ae2fa 100644 --- a/t/meson.build +++ b/t/meson.build @@ -1210,12 +1210,18 @@ test_environment = script_environment test_environment.set('GIT_BUILD_DIR', git_build_dir) foreach integration_test : integration_tests + per_test_kwargs = test_kwargs + # The fsmonitor tests start daemon processes that in some environments + # can hang. Set a generous timeout to prevent CI from blocking. + if fs.stem(integration_test) == 't7527-builtin-fsmonitor' + per_test_kwargs += {'timeout': 1800} + endif test(fs.stem(integration_test), shell, args: [ integration_test ], workdir: meson.current_source_dir(), env: test_environment, depends: test_dependencies + bin_wrappers, - kwargs: test_kwargs, + kwargs: per_test_kwargs, ) endforeach diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..774da5ac60 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -10,9 +10,58 @@ then test_done fi +# Verify that the filesystem delivers events to the daemon. +# On some configurations (e.g., overlayfs with older kernels), +# inotify watches succeed but events are never delivered. The +# cookie wait will time out and the daemon logs a trace message. +# +# Use "timeout" (if available) to guard each step against hangs. +maybe_timeout () { + if type timeout >/dev/null 2>&1 + then + timeout "$@" + else + shift + "$@" + fi +} +verify_fsmonitor_works () { + git init test_fsmonitor_smoke || return 1 + + GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && + export GIT_TRACE_FSMONITOR && + maybe_timeout 30 \ + git -C test_fsmonitor_smoke fsmonitor--daemon start \ + --start-timeout=10 + ret=$? + unset GIT_TRACE_FSMONITOR + if test $ret -ne 0 + then + rm -rf test_fsmonitor_smoke smoke.trace + return 1 + fi + + maybe_timeout 10 \ + test-tool -C test_fsmonitor_smoke fsmonitor-client query \ + --token 0 >/dev/null 2>&1 + maybe_timeout 5 \ + git -C test_fsmonitor_smoke fsmonitor--daemon stop 2>/dev/null + ! grep -q "cookie_wait timed out" "$PWD/smoke.trace" 2>/dev/null + ret=$? + rm -rf test_fsmonitor_smoke smoke.trace + return $ret +} + +if ! verify_fsmonitor_works +then + skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" + test_done +fi + stop_daemon_delete_repo () { r=$1 && - test_might_fail git -C $r fsmonitor--daemon stop && + test_might_fail maybe_timeout 30 \ + git -C $r fsmonitor--daemon stop 2>/dev/null rm -rf $1 } @@ -67,7 +116,7 @@ start_daemon () { export GIT_TEST_FSMONITOR_TOKEN fi && - git $r fsmonitor--daemon start && + git $r fsmonitor--daemon start --start-timeout=10 && git $r fsmonitor--daemon status ) } @@ -520,6 +569,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the @@ -910,7 +981,10 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" ' start_git_in_background () { git "$@" & git_pid=$! - git_pgid=$(ps -o pgid= -p $git_pid) + git_pgid=$(ps -o pgid= -p $git_pid 2>/dev/null || + awk '{print $5}' /proc/$git_pid/stat 2>/dev/null) && + git_pgid="${git_pgid## }" && + git_pgid="${git_pgid%% }" nr_tries_left=10 while true do @@ -921,15 +995,16 @@ start_git_in_background () { fi sleep 1 nr_tries_left=$(($nr_tries_left - 1)) - done >/dev/null 2>&1 & + done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! wait $git_pid } stop_git () { - while kill -0 -- -$git_pgid + test -n "$git_pgid" || return 0 + while kill -0 -- -$git_pgid 2>/dev/null do - kill -- -$git_pgid + kill -- -$git_pgid 2>/dev/null sleep 1 done } @@ -944,7 +1019,7 @@ stop_watchdog () { test_expect_success !MINGW "submodule implicitly starts daemon by pull" ' test_atexit "stop_watchdog" && - test_when_finished "stop_git; rm -rf cloned super sub" && + test_when_finished "set +m; stop_git; rm -rf cloned super sub" && create_super super && create_sub sub && -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v6 09/10] run-command: add close_fd_above_stderr option 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (7 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 08/10] fsmonitor: add tests " Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-25 21:41 ` Junio C Hamano 2026-02-25 20:17 ` [PATCH v6 10/10] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 10 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a new option to struct child_process that closes file descriptors 3 and above in the child after forking but before exec. This prevents long-running child processes from inheriting pipe endpoints or other descriptors from the parent environment. The upper bound for the fd scan comes from sysconf(_SC_OPEN_MAX), capped at 4096 to avoid excessive iteration when the limit is set very high. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- run-command.c | 11 +++++++++++ run-command.h | 9 +++++++++ 2 files changed, 20 insertions(+) diff --git a/run-command.c b/run-command.c index e3e02475cc..cbadcf5ff8 100644 --- a/run-command.c +++ b/run-command.c @@ -832,6 +832,17 @@ fail_pipe: child_close(cmd->out); } + if (cmd->close_fd_above_stderr) { + long max_fd = sysconf(_SC_OPEN_MAX); + int fd; + if (max_fd < 0 || max_fd > 4096) + max_fd = 4096; + for (fd = 3; fd < max_fd; fd++) { + if (fd != child_notifier) + close(fd); + } + } + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); diff --git a/run-command.h b/run-command.h index 0df25e445f..a1aa1b1069 100644 --- a/run-command.h +++ b/run-command.h @@ -141,6 +141,15 @@ struct child_process { unsigned stdout_to_stderr:1; unsigned clean_on_exit:1; unsigned wait_after_clean:1; + + /** + * Close file descriptors 3 and above in the child after forking + * but before exec. This prevents the long-running child from + * inheriting pipe endpoints or other descriptors from the parent + * environment (e.g., the test harness). + */ + unsigned close_fd_above_stderr:1; + void (*clean_on_exit_handler)(struct child_process *process); }; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v6 09/10] run-command: add close_fd_above_stderr option 2026-02-25 20:17 ` [PATCH v6 09/10] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget @ 2026-02-25 21:41 ` Junio C Hamano 0 siblings, 0 replies; 129+ messages in thread From: Junio C Hamano @ 2026-02-25 21:41 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Patrick Steinhardt, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Paul Tarjan <github@paulisageek.com> > > Add a new option to struct child_process that closes file descriptors > 3 and above in the child after forking but before exec. This prevents > long-running child processes from inheriting pipe endpoints or other > descriptors from the parent environment. > > The upper bound for the fd scan comes from sysconf(_SC_OPEN_MAX), > capped at 4096 to avoid excessive iteration when the limit is set > very high. > > Signed-off-by: Paul Tarjan <github@paulisageek.com> > --- > run-command.c | 11 +++++++++++ > run-command.h | 9 +++++++++ > 2 files changed, 20 insertions(+) All makes sense. I somehow find it a bit surprising that we did not already have this feature anywhere, as closing all except for the low file descriptors connected to stdio by default is fairly a common thing to do. > diff --git a/run-command.c b/run-command.c > index e3e02475cc..cbadcf5ff8 100644 > --- a/run-command.c > +++ b/run-command.c > @@ -832,6 +832,17 @@ fail_pipe: > child_close(cmd->out); > } > > + if (cmd->close_fd_above_stderr) { > + long max_fd = sysconf(_SC_OPEN_MAX); > + int fd; > + if (max_fd < 0 || max_fd > 4096) > + max_fd = 4096; > + for (fd = 3; fd < max_fd; fd++) { > + if (fd != child_notifier) > + close(fd); > + } > + } > + > if (cmd->dir && chdir(cmd->dir)) > child_die(CHILD_ERR_CHDIR); > > diff --git a/run-command.h b/run-command.h > index 0df25e445f..a1aa1b1069 100644 > --- a/run-command.h > +++ b/run-command.h > @@ -141,6 +141,15 @@ struct child_process { > unsigned stdout_to_stderr:1; > unsigned clean_on_exit:1; > unsigned wait_after_clean:1; > + > + /** > + * Close file descriptors 3 and above in the child after forking > + * but before exec. This prevents the long-running child from > + * inheriting pipe endpoints or other descriptors from the parent > + * environment (e.g., the test harness). > + */ > + unsigned close_fd_above_stderr:1; > + > void (*clean_on_exit_handler)(struct child_process *process); > }; ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v6 10/10] fsmonitor: close inherited file descriptors and detach in daemon 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (8 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 09/10] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 ` Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 10 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-25 20:17 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> When the fsmonitor daemon is spawned as a background process, it may inherit file descriptors from its parent that it does not need. In particular, when the test harness or a CI system captures output through pipes, the daemon can inherit duplicated pipe endpoints. If the daemon holds these open, the parent process never sees EOF and may appear to hang. Set close_fd_above_stderr on the child process at daemon startup so that file descriptors 3 and above are closed before any daemon work begins. This ensures the daemon does not inadvertently hold open descriptors from its launching environment. Additionally, call setsid() when the daemon starts with --detach to create a new session and process group. Without this, shells that enable job control (e.g. bash with "set -m") treat the daemon as part of the spawning command's job. Their "wait" builtin then blocks until the daemon exits, which it never does. This specifically affects systems where /bin/sh is bash (e.g. Fedora), since dash only waits for the specific PID rather than the full process group. Add a 30-second timeout to "fsmonitor--daemon stop" so it does not block indefinitely if the daemon fails to shut down. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 28 +++++++++++++++++++++++++--- fsmonitor-ipc.c | 3 +++ t/meson.build | 8 +------- t/t7527-builtin-fsmonitor.sh | 12 +++++++++++- 4 files changed, 40 insertions(+), 11 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index f6c406ff12..4ed848e79e 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -86,6 +86,8 @@ static int do_as_client__send_stop(void) { struct strbuf answer = STRBUF_INIT; int ret; + int max_wait_ms = 30000; + int elapsed_ms = 0; ret = fsmonitor_ipc__send_command("quit", &answer); @@ -96,8 +98,16 @@ static int do_as_client__send_stop(void) return ret; trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL); - while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) + while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) { + if (elapsed_ms >= max_wait_ms) { + trace2_region_leave("fsm_client", + "polling-for-daemon-exit", NULL); + return error(_("daemon did not stop within %d seconds"), + max_wait_ms / 1000); + } sleep_millisec(50); + elapsed_ms += 50; + } trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL); return 0; @@ -1431,7 +1441,7 @@ done: return err; } -static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) +static int try_to_run_foreground_daemon(int detach_console) { /* * Technically, we don't need to probe for an existing daemon @@ -1451,10 +1461,21 @@ static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) fflush(stderr); } + if (detach_console) { #ifdef GIT_WINDOWS_NATIVE - if (detach_console) FreeConsole(); +#else + /* + * Create a new session so that the daemon is detached + * from the parent's process group. This prevents + * shells with job control (e.g. bash with "set -m") + * from waiting on the daemon when they wait for a + * foreground command that implicitly spawned it. + */ + if (setsid() == -1) + warning_errno(_("setsid failed")); #endif + } return !!fsmonitor_run_daemon(); } @@ -1517,6 +1538,7 @@ static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; + cp.close_fd_above_stderr = 1; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c index f1b1631111..6112d13064 100644 --- a/fsmonitor-ipc.c +++ b/fsmonitor-ipc.c @@ -61,6 +61,9 @@ static int spawn_daemon(void) cmd.git_cmd = 1; cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; + cmd.close_fd_above_stderr = 1; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); diff --git a/t/meson.build b/t/meson.build index 85ef2ae2fa..19e8306298 100644 --- a/t/meson.build +++ b/t/meson.build @@ -1210,18 +1210,12 @@ test_environment = script_environment test_environment.set('GIT_BUILD_DIR', git_build_dir) foreach integration_test : integration_tests - per_test_kwargs = test_kwargs - # The fsmonitor tests start daemon processes that in some environments - # can hang. Set a generous timeout to prevent CI from blocking. - if fs.stem(integration_test) == 't7527-builtin-fsmonitor' - per_test_kwargs += {'timeout': 1800} - endif test(fs.stem(integration_test), shell, args: [ integration_test ], workdir: meson.current_source_dir(), env: test_environment, depends: test_dependencies + bin_wrappers, - kwargs: per_test_kwargs, + kwargs: test_kwargs, ) endforeach diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 774da5ac60..d7e64bcb7a 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -766,7 +766,7 @@ do else test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] enable fsmonitor" ' git config core.fsmonitor true && - git fsmonitor--daemon start && + git fsmonitor--daemon start --start-timeout=10 && git update-index --fsmonitor ' fi @@ -997,7 +997,17 @@ start_git_in_background () { nr_tries_left=$(($nr_tries_left - 1)) done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! + + # Disable job control before wait. With "set -m", bash treats + # "wait $pid" as waiting for the entire job (process group), + # which blocks indefinitely if the fsmonitor daemon was spawned + # into the same process group and is still running. Turning off + # job control makes "wait" only wait for the specific PID. + set +m && wait $git_pid + wait_status=$? + set -m + return $wait_status } stop_git () { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget ` (9 preceding siblings ...) 2026-02-25 20:17 ` [PATCH v6 10/10] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget ` (11 more replies) 10 siblings, 12 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan This series implements the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. It also fixes two memory leaks in the platform-independent daemon code and deduplicates the IPC and settings logic that is now shared between macOS and Linux. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. This series builds on work from https://github.com/git/git/pull/1352 by Eric DeCosta and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v6: * Introduced FSMONITOR_OS_SETTINGS build variable (set to "unix" for macOS and Linux, "win32" for Windows) to eliminate if/else conditionals in Makefile, meson.build, and CMakeLists.txt per Junio's review * Moved fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND since path-utils files are platform-specific * Removed V9FS_MAGIC from remote filesystem detection (9p is used for local VM/container host mounts where fsmonitor works fine) * Removed redundant #include <libgen.h> (already provided by compat/posix.h) * Fixed cookie wait comment wording ("see" → "observe") * Rewrote commit messages for IPC and settings dedup patches Changes since v5: * Split monolithic commit into 10-patch series per Patrick's review * Deduplicated fsm-ipc and fsm-settings into shared Unix implementations * Rewrote commit message with prose paragraphs, explain inotify vs fanotify, removed "Issues addressed" sections, added Based-on-patch-by trailers * Removed redundant includes already provided by compat/posix.h * Fixed error/trace message capitalization per coding guidelines * Fixed stale rename check interval from 1000 seconds to 1 second * Changed poll timeout from 1ms to 50ms to reduce idle CPU wake-ups * Replaced infinite pthread_cond_wait cookie loop with one-second pthread_cond_timedwait (prevents daemon hangs on overlay filesystems where events are never delivered) * Added pthread_cond_timedwait to Windows pthread compatibility layer * Separated test into its own commit with smoke test that skips when inotify events are not delivered (e.g., overlayfs with older kernels) * Fixed test hang on Fedora CI: stop_git() looped forever when ps was unavailable because bash in POSIX/sh mode returns exit 0 from kill with an empty process group argument. Fixed by falling back to /proc/$pid/stat for process group ID and guarding stop_git against empty pgid. * Redirect spawn_daemon() stdout/stderr to /dev/null and close inherited file descriptors to prevent the intermediate process from holding test pipe file descriptors * Call setsid() on daemon detach to prevent shells with job control from waiting on the daemon process group * Close inherited file descriptors 3-7 in the test watchdog subprocess * Added 30-second timeout to "fsmonitor--daemon stop" to prevent indefinite blocking * Added helpful error message when inotify watch limit (max_user_watches) is reached * Initialize fd_inotify to -1 and use fd >= 0 check for correct fd 0 handling * Use sysconf(_SC_OPEN_MAX) instead of hardcoded 1024 for fd close limit * Check setsid() return value Changes since v4: * Added Meson build support Changes since v3: * Fix crash on rapid nested directory creation (EEXIST from inotify_add_watch with IN_MASK_CREATE) * Extensive stress testing Changes since v2: * Fix khash memory leak in do_handle_client Changes since v1: * Fix hashmap memory leak in fsmonitor_run_daemon() Paul Tarjan (10): fsmonitor: fix khash memory leak in do_handle_client fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon compat/win32: add pthread_cond_timedwait fsmonitor: use pthread_cond_timedwait for cookie wait fsmonitor: deduplicate IPC path logic for Unix platforms fsmonitor: deduplicate settings logic for Unix platforms fsmonitor: implement filesystem change listener for Linux fsmonitor: add tests for Linux run-command: add close_fd_above_stderr option fsmonitor: close inherited file descriptors and detach in daemon Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- Makefile | 4 +- builtin/fsmonitor--daemon.c | 71 +- compat/fsmonitor/fsm-health-linux.c | 33 + .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 4 +- compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 220 ++++++ ...-settings-darwin.c => fsm-settings-unix.c} | 24 +- compat/win32/pthread.c | 26 + compat/win32/pthread.h | 2 + config.mak.uname | 12 +- contrib/buildsystems/CMakeLists.txt | 29 +- fsmonitor-ipc.c | 3 + meson.build | 13 +- run-command.c | 11 + run-command.h | 9 + t/t7527-builtin-fsmonitor.sh | 101 ++- 18 files changed, 1278 insertions(+), 62 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (96%) create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (82%) base-commit: 3e0db84c88c57e70ac8be8c196dfa92c5d656fbc Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v7 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v7 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v6: 1: 4d4dec8fa1 = 1: 4d4dec8fa1 fsmonitor: fix khash memory leak in do_handle_client 2: d0bd3e32ca = 2: d0bd3e32ca fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 3: d2c5ca0939 = 3: d2c5ca0939 compat/win32: add pthread_cond_timedwait 4: 0051a19303 ! 4: 0a58670952 fsmonitor: use pthread_cond_timedwait for cookie wait @@ builtin/fsmonitor--daemon.c: static enum fsmonitor_cookie_item_result with_lock_ - * this. Also note that the above open() and unlink() calls - * will cause at least two FS events on that path, so the odds - * of getting stuck are pretty slim. -+ * Wait for the listener thread to see the cookie file. ++ * Wait for the listener thread to observe the cookie file. + * Time out after a short interval so that the client + * does not hang forever if the filesystem does not deliver + * events (e.g., on certain container/overlay filesystems 5: ff31e359a7 ! 5: 037ae2a03f fsmonitor: deduplicate IPC path logic for Unix platforms @@ Metadata ## Commit message ## fsmonitor: deduplicate IPC path logic for Unix platforms - The IPC path logic for determining the Unix domain socket location is - nearly identical between macOS and Linux. Both need to check whether - the .git directory is on a remote filesystem and, if so, fall back to - a socket path under $HOME or a user-configured directory. + The macOS fsm-ipc-darwin.c is applicable to other Unix variants as + well. Rename it to fsm-ipc-unix.c and add a worktree NULL check + (BUG guard) that was missing from the macOS version. - Merge the two implementations into a single fsm-ipc-unix.c that is - shared by both platforms. The unified version includes the worktree - NULL check (BUG guard) from the Linux implementation, which was missing - in the macOS version. - - Update Makefile, meson.build, and CMakeLists.txt to use the new shared - file for non-Windows platforms. + To support this, introduce FSMONITOR_OS_SETTINGS which is set to + "unix" for both macOS and Linux, distinct from FSMONITOR_DAEMON_BACKEND + which remains platform-specific (darwin, linux, win32). Move + fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND + since the path-utils files are platform-specific. Signed-off-by: Paul Tarjan <github@paulisageek.com> @@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND endif ifdef FSMONITOR_OS_SETTINGS + COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o +- COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o ++ COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o + endif + + ifdef WITH_BREAKING_CHANGES ## compat/fsmonitor/fsm-ipc-darwin.c => compat/fsmonitor/fsm-ipc-unix.c ## @@ compat/fsmonitor/fsm-ipc-unix.c: const char *fsmonitor_ipc__get_path(struct repository *r) 6: 39da1e6be3 ! 6: 0a83bb9c8e fsmonitor: deduplicate settings logic for Unix platforms @@ Metadata ## Commit message ## fsmonitor: deduplicate settings logic for Unix platforms - The fsmonitor settings logic for checking Unix domain socket - compatibility is nearly identical between macOS and Linux. Both check - whether the IPC socket directory resides on a remote filesystem or a - filesystem that does not support Unix domain sockets (NTFS, FAT32, etc). + The macOS fsm-settings-darwin.c is applicable to other Unix variants + as well. Rename it to fsm-settings-unix.c, using the safer + xstrdup()+dirname() approach and including the "vfat" filesystem check. - Merge the two implementations into a single fsm-settings-unix.c shared - by both platforms. The unified version uses the safer xstrdup() + - dirname() approach from the macOS implementation (avoiding strbuf - mutation with dirname()) and includes the "vfat" filesystem check. - - Update Makefile, meson.build, and CMakeLists.txt to use the new shared - file for non-Windows platforms. + Now that both fsm-ipc and fsm-settings use the "unix" variant name, + set FSMONITOR_OS_SETTINGS to "unix" for macOS in config.mak.uname and + remove the if/else conditionals from the build files. Signed-off-by: Paul Tarjan <github@paulisageek.com> ## Makefile ## -@@ Makefile: endif +@@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND + COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND + COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o + COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o +-ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) +- COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o +-else +- COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o +-endif + endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS -- COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o -+ifeq ($(FSMONITOR_OS_SETTINGS),win32) -+ COMPAT_OBJS += compat/fsmonitor/fsm-settings-win32.o -+else -+ COMPAT_OBJS += compat/fsmonitor/fsm-settings-unix.o -+endif - COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o ++ COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif - ## compat/fsmonitor/fsm-settings-darwin.c => compat/fsmonitor/fsm-settings-unix.c ## @@ @@ compat/fsmonitor/fsm-settings-darwin.c => compat/fsmonitor/fsm-settings-unix.c return FSMONITOR_REASON_NOSOCKETS; } + ## config.mak.uname ## +@@ config.mak.uname: ifeq ($(uname_S),Darwin) + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = darwin +- FSMONITOR_OS_SETTINGS = darwin ++ FSMONITOR_OS_SETTINGS = unix + endif + endif + + ## contrib/buildsystems/CMakeLists.txt ## -@@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) +@@ contrib/buildsystems/CMakeLists.txt: endif() + + if(SUPPORTS_SIMPLE_IPC) + if(CMAKE_SYSTEM_NAME STREQUAL "Windows") +- add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) +- +- add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) ++ set(FSMONITOR_DAEMON_BACKEND "win32") ++ set(FSMONITOR_OS_SETTINGS "win32") + elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") ++ set(FSMONITOR_DAEMON_BACKEND "darwin") ++ set(FSMONITOR_OS_SETTINGS "unix") ++ endif() ++ ++ if(FSMONITOR_DAEMON_BACKEND) + add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-${FSMONITOR_DAEMON_BACKEND}.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-${FSMONITOR_DAEMON_BACKEND}.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-${FSMONITOR_OS_SETTINGS}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) endif() endif() ## meson.build ## +@@ meson.build: else + endif + + fsmonitor_backend = '' ++fsmonitor_os = '' + if host_machine.system() == 'windows' + fsmonitor_backend = 'win32' ++ fsmonitor_os = 'win32' + elif host_machine.system() == 'darwin' + fsmonitor_backend = 'darwin' ++ fsmonitor_os = 'unix' + libgit_dependencies += dependency('CoreServices') + endif + if fsmonitor_backend != '' @@ meson.build: if fsmonitor_backend != '' 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ++ 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', ++ 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', ] - - if fsmonitor_backend == 'win32' +- +- if fsmonitor_backend == 'win32' - libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' -+ libgit_sources += [ -+ 'compat/fsmonitor/fsm-ipc-win32.c', -+ 'compat/fsmonitor/fsm-settings-win32.c', -+ ] - else +- else - libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' -+ libgit_sources += [ -+ 'compat/fsmonitor/fsm-ipc-unix.c', -+ 'compat/fsmonitor/fsm-settings-unix.c', -+ ] - endif +- endif endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) +-build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) ++build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_os) + + if not get_option('b_sanitize').contains('address') and get_option('regex').allowed() and compiler.has_header('regex.h') and compiler.get_define('REG_STARTEND', prefix: '#include <regex.h>') != '' + build_options_config.set('NO_REGEX', '') 7: 4eadc06004 ! 7: adac7964cc fsmonitor: implement filesystem change listener for Linux @@ config.mak.uname: ifeq ($(uname_S),Linux) + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux -+ FSMONITOR_OS_SETTINGS = linux ++ FSMONITOR_OS_SETTINGS = unix + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif @@ config.mak.uname: ifeq ($(uname_S),Linux) ## contrib/buildsystems/CMakeLists.txt ## @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) - -+ add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) + elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") + set(FSMONITOR_DAEMON_BACKEND "darwin") + set(FSMONITOR_OS_SETTINGS "unix") + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") -+ add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) ++ set(FSMONITOR_DAEMON_BACKEND "linux") ++ set(FSMONITOR_OS_SETTINGS "unix") + add_compile_definitions(HAVE_LINUX_MAGIC_H) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-linux.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-linux.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-linux.c) -+ - add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-unix.c) endif() + + if(FSMONITOR_DAEMON_BACKEND) ## meson.build ## -@@ meson.build: endif - fsmonitor_backend = '' +@@ meson.build: fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' + fsmonitor_os = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' ++ fsmonitor_os = 'unix' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' - libgit_dependencies += dependency('CoreServices') + fsmonitor_os = 'unix' 8: 8fec92d5b4 = 8: fad2f0a81a fsmonitor: add tests for Linux 9: 817489b3ea = 9: c684fc9094 run-command: add close_fd_above_stderr option 10: bb438afbbe = 10: 4987a009a2 fsmonitor: close inherited file descriptors and detach in daemon -- gitgitgadget ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget ` (10 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `shown` kh_str_t was freed with kh_release_str() at a point in the code only reachable in the non-trivial response path. When the client receives a trivial response, the code jumps to the `cleanup` label, skipping the kh_release_str() call entirely and leaking the hash table. Fix this by initializing `shown` to NULL and moving the cleanup to the `cleanup` label using kh_destroy_str(), which is safe to call on NULL. This ensures the hash table is freed regardless of which code path is taken. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..bc4571938c 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client 2026-02-26 0:27 ` [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:42 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:14AM +0000, Paul Tarjan via GitGitGadget wrote: > diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c > index 242c594646..bc4571938c 100644 > --- a/builtin/fsmonitor--daemon.c > +++ b/builtin/fsmonitor--daemon.c > @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > const struct fsmonitor_batch *batch; > struct fsmonitor_batch *remainder = NULL; > intmax_t count = 0, duplicates = 0; > - kh_str_t *shown; > + kh_str_t *shown = NULL; > int hash_ret; > int do_trivial = 0; > int do_flush = 0; > @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > total_response_len += payload.len; > } > > - kh_release_str(shown); > - > pthread_mutex_lock(&state->main_lock); > > if (token_data->client_ref_count > 0) > @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, > trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); > > cleanup: > + kh_destroy_str(shown); > strbuf_release(&response_token); > strbuf_release(&requested_token_id); > strbuf_release(&payload); Makes sense. If I understood correctly I think we could improve this code to stop using khash directly and instead use a strmap, which has a nicer interface. But that's certainly outside of the scope of this patch series and rather a #leftoverbit. Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client 2026-03-04 7:42 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > Makes sense. If I understood correctly I think we could improve this > code to stop using khash directly and instead use a strmap, which > has a nicer interface. But that's certainly outside of the scope of this > patch series and rather a #leftoverbit. Went ahead and did this in v8 as patch 12. Switched to strset (since we only need a set, not a map). Ended up being a nice simplification: strset_add() returns whether the entry is new, so the lookup+insert becomes a single call. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 03/10] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget ` (9 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `state.cookies` hashmap is initialized during daemon startup but never freed during cleanup in the `done:` label of fsmonitor_run_daemon(). Add a hashmap_clear() call to prevent this memory leak. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 1 + 1 file changed, 1 insertion(+) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index bc4571938c..4d52622e24 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1404,6 +1404,7 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + hashmap_clear(&state.cookies); fsm_listen__dtor(&state); fsm_health__dtor(&state); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-02-26 0:27 ` [PATCH v7 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget @ 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:42 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:15AM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> > > The `state.cookies` hashmap is initialized during daemon startup but > never freed during cleanup in the `done:` label of > fsmonitor_run_daemon(). Add a hashmap_clear() call to prevent this > memory leak. > > Signed-off-by: Paul Tarjan <github@paulisageek.com> > --- > builtin/fsmonitor--daemon.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c > index bc4571938c..4d52622e24 100644 > --- a/builtin/fsmonitor--daemon.c > +++ b/builtin/fsmonitor--daemon.c > @@ -1404,6 +1404,7 @@ static int fsmonitor_run_daemon(void) > done: > pthread_cond_destroy(&state.cookies_cond); > pthread_mutex_destroy(&state.main_lock); > + hashmap_clear(&state.cookies); > fsm_listen__dtor(&state); > fsm_health__dtor(&state); Is this actually sufficient? as far as I can see, the cookies are inserted in `__wait_for_cookie()`, and each cookie also has a name attached to it that was allocated via a strbuf. So don't we have to free the name, as well? Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-03-04 7:42 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > Is this actually sufficient? as far as I can see, the cookies are > inserted in `__wait_for_cookie()`, and each cookie also has a name > attached to it that was allocated via a strbuf. So don't we have to > free the name, as well? You're right, the cookie names come from strbuf_detach() so they need to be freed too. Fixed in v8: iterates and frees each name before calling hashmap_clear_and_free(). ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 03/10] compat/win32: add pthread_cond_timedwait 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget ` (8 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pthread_cond_timedwait() implementation to the Windows pthread compatibility layer using SleepConditionVariableCS() with a millisecond timeout computed from the absolute deadline. This enables callers to use bounded waits on condition variables instead of blocking indefinitely with pthread_cond_wait(). Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/win32/pthread.c | 26 ++++++++++++++++++++++++++ compat/win32/pthread.h | 2 ++ 2 files changed, 28 insertions(+) diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c index 7e93146963..538ef92d9d 100644 --- a/compat/win32/pthread.c +++ b/compat/win32/pthread.c @@ -66,3 +66,29 @@ int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) return err_win_to_posix(GetLastError()); return 0; } + +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime) +{ + struct timeval now; + long long now_ms, deadline_ms; + DWORD timeout_ms; + + gettimeofday(&now, NULL); + now_ms = (long long)now.tv_sec * 1000 + now.tv_usec / 1000; + deadline_ms = (long long)abstime->tv_sec * 1000 + + abstime->tv_nsec / 1000000; + + if (deadline_ms <= now_ms) + timeout_ms = 0; + else + timeout_ms = (DWORD)(deadline_ms - now_ms); + + if (SleepConditionVariableCS(cond, mutex, timeout_ms) == 0) { + DWORD err = GetLastError(); + if (err == ERROR_TIMEOUT) + return ETIMEDOUT; + return err_win_to_posix(err); + } + return 0; +} diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index ccacc5a53b..d80df8d12a 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -64,6 +64,8 @@ int win32_pthread_join(pthread_t *thread, void **value_ptr); pthread_t pthread_self(void); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime); static inline void NORETURN pthread_exit(void *ret) { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 03/10] compat/win32: add pthread_cond_timedwait 2026-02-26 0:27 ` [PATCH v7 03/10] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget @ 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:42 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:16AM +0000, Paul Tarjan via GitGitGadget wrote: > diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c > index 7e93146963..538ef92d9d 100644 > --- a/compat/win32/pthread.c > +++ b/compat/win32/pthread.c > @@ -66,3 +66,29 @@ int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) > return err_win_to_posix(GetLastError()); > return 0; > } > + > +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, > + const struct timespec *abstime) > +{ > + struct timeval now; > + long long now_ms, deadline_ms; > + DWORD timeout_ms; > + > + gettimeofday(&now, NULL); > + now_ms = (long long)now.tv_sec * 1000 + now.tv_usec / 1000; > + deadline_ms = (long long)abstime->tv_sec * 1000 + > + abstime->tv_nsec / 1000000; > + > + if (deadline_ms <= now_ms) > + timeout_ms = 0; According to pthread_cond_timedwait(3p) we should return an error in thas case: The pthread_cond_timedwait() function shall be equivalent to pthread_cond_wait(), except that an error is returned if the absolute time specified by abstime passes (that is, system time equals or exceeds abstime) before the condition cond is signaled or broadcasted, or if the absolute time specified by abstime has already been passed at the time of the call. So I guess it's safe to return ETIMEDOUT directly here? > + else > + timeout_ms = (DWORD)(deadline_ms - now_ms); > + > + if (SleepConditionVariableCS(cond, mutex, timeout_ms) == 0) { The function returns a BOOL, so comparing with `== 0` is misleading. I see that this is following the pattern of `pthread_cond_wait()` though, so I guess it's okayish. > + DWORD err = GetLastError(); > + if (err == ERROR_TIMEOUT) > + return ETIMEDOUT; > + return err_win_to_posix(err); Wouldn't it make sense to extend `err_win_to_posix()` instead? Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 03/10] compat/win32: add pthread_cond_timedwait 2026-03-04 7:42 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > So I guess it's safe to return ETIMEDOUT directly here? Yep, that matches the POSIX spec. Fixed in v8. > Wouldn't it make sense to extend `err_win_to_posix()` instead? err_win_to_posix() currently maps ERROR_TIMEOUT to EBUSY, not ETIMEDOUT. Changing that mapping could affect other callers, so it seemed safer to handle WAIT_TIMEOUT explicitly in this function. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (2 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 03/10] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget ` (7 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The cookie wait in with_lock__wait_for_cookie() uses an infinite pthread_cond_wait() loop. The existing comment notes the desire to switch to pthread_cond_timedwait(), but the routine was not available in git thread-utils. On certain container or overlay filesystems, inotify watches may succeed but events are never delivered. In this case the daemon would hang indefinitely waiting for the cookie event, which in turn causes the client to hang. Replace the infinite wait with a one-second timeout using pthread_cond_timedwait(). If the timeout fires, report an error and let the client proceed with a trivial (full-scan) response rather than blocking forever. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 4d52622e24..110fe5fb55 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -197,20 +197,31 @@ static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie( unlink(cookie_pathname.buf); /* - * Technically, this is an infinite wait (well, unless another - * thread sends us an abort). I'd like to change this to - * use `pthread_cond_timedwait()` and return an error/timeout - * and let the caller do the trivial response thing, but we - * don't have that routine in our thread-utils. - * - * After extensive beta testing I'm not really worried about - * this. Also note that the above open() and unlink() calls - * will cause at least two FS events on that path, so the odds - * of getting stuck are pretty slim. + * Wait for the listener thread to observe the cookie file. + * Time out after a short interval so that the client + * does not hang forever if the filesystem does not deliver + * events (e.g., on certain container/overlay filesystems + * where inotify watches succeed but events never arrive). */ - while (cookie->result == FCIR_INIT) - pthread_cond_wait(&state->cookies_cond, - &state->main_lock); + { + struct timeval now; + struct timespec ts; + int err = 0; + + gettimeofday(&now, NULL); + ts.tv_sec = now.tv_sec + 1; + ts.tv_nsec = now.tv_usec * 1000; + + while (cookie->result == FCIR_INIT && !err) + err = pthread_cond_timedwait(&state->cookies_cond, + &state->main_lock, + &ts); + if (err == ETIMEDOUT && cookie->result == FCIR_INIT) { + trace_printf_key(&trace_fsmonitor, + "cookie_wait timed out"); + cookie->result = FCIR_ERROR; + } + } done: hashmap_remove(&state->cookies, &cookie->entry, NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-02-26 0:27 ` [PATCH v7 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:42 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:17AM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> > > The cookie wait in with_lock__wait_for_cookie() uses an infinite > pthread_cond_wait() loop. The existing comment notes the desire > to switch to pthread_cond_timedwait(), but the routine was not > available in git thread-utils. > > On certain container or overlay filesystems, inotify watches may > succeed but events are never delivered. In this case the daemon > would hang indefinitely waiting for the cookie event, which in > turn causes the client to hang. > > Replace the infinite wait with a one-second timeout using > pthread_cond_timedwait(). If the timeout fires, report an > error and let the client proceed with a trivial (full-scan) > response rather than blocking forever. One thing that I'd be happy to learn about is why specifically you have chosen one second as a timeout value. Are we sure this is always enough on a loaded system? Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-03-04 7:42 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > One thing that I'd be happy to learn about is why specifically you have > chosen one second as a timeout value. Are we sure this is always enough > on a loaded system? The cookie round-trip (create temp file, get inotify event) is normally a few milliseconds on a local filesystem, so 1 second gives plenty of headroom. If it does fire on a heavily loaded box, the fallback is a trivial (full-scan) response, same as any other error path, so the user still gets correct results, just with a bit more work on their end. Going much longer means the client just sits there waiting when something is actually broken (like overlayfs not delivering events), which is worse. Open to bumping it if you think 1s is too aggressive though. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (3 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget ` (6 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The macOS fsm-ipc-darwin.c is applicable to other Unix variants as well. Rename it to fsm-ipc-unix.c and add a worktree NULL check (BUG guard) that was missing from the macOS version. To support this, introduce FSMONITOR_OS_SETTINGS which is set to "unix" for both macOS and Linux, distinct from FSMONITOR_DAEMON_BACKEND which remains platform-specific (darwin, linux, win32). Move fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND since the path-utils files are platform-specific. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 8 ++++++-- compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 4 +++- contrib/buildsystems/CMakeLists.txt | 2 +- meson.build | 7 ++++++- 4 files changed, 16 insertions(+), 5 deletions(-) rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (96%) diff --git a/Makefile b/Makefile index 8aa489f3b6..7480ce3e1d 100644 --- a/Makefile +++ b/Makefile @@ -2365,13 +2365,17 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o +ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o +else + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o +endif endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o - COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef WITH_BREAKING_CHANGES diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-unix.c similarity index 96% rename from compat/fsmonitor/fsm-ipc-darwin.c rename to compat/fsmonitor/fsm-ipc-unix.c index fe149a1b37..d34a6419bc 100644 --- a/compat/fsmonitor/fsm-ipc-darwin.c +++ b/compat/fsmonitor/fsm-ipc-unix.c @@ -27,13 +27,15 @@ const char *fsmonitor_ipc__get_path(struct repository *r) if (ipc_path) return ipc_path; - /* By default the socket file is created in the .git directory */ if (fsmonitor__is_fs_remote(r->gitdir) < 1) { ipc_path = fsmonitor_ipc__get_default_path(); return ipc_path; } + if (!r->worktree) + BUG("repository has no worktree"); + git_SHA1_Init(&sha1ctx); git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); git_SHA1_Final(hash, &sha1ctx); diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..32ef6ebe1b 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -303,7 +303,7 @@ if(SUPPORTS_SIMPLE_IPC) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) diff --git a/meson.build b/meson.build index dd52efd1c8..8de795f9d4 100644 --- a/meson.build +++ b/meson.build @@ -1332,11 +1332,16 @@ if fsmonitor_backend != '' libgit_sources += [ 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] + + if fsmonitor_backend == 'win32' + libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' + else + libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' + endif endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms 2026-02-26 0:27 ` [PATCH v7 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget @ 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:42 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:18AM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> Nit: we're not exactly deduplicating just yet, but are rather preparing for that as there is no second implementation using this yet. > diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-unix.c > similarity index 96% > rename from compat/fsmonitor/fsm-ipc-darwin.c > rename to compat/fsmonitor/fsm-ipc-unix.c > index fe149a1b37..d34a6419bc 100644 > --- a/compat/fsmonitor/fsm-ipc-darwin.c > +++ b/compat/fsmonitor/fsm-ipc-unix.c > @@ -27,13 +27,15 @@ const char *fsmonitor_ipc__get_path(struct repository *r) > if (ipc_path) > return ipc_path; > > - > /* By default the socket file is created in the .git directory */ > if (fsmonitor__is_fs_remote(r->gitdir) < 1) { > ipc_path = fsmonitor_ipc__get_default_path(); > return ipc_path; > } > > + if (!r->worktree) > + BUG("repository has no worktree"); > + > git_SHA1_Init(&sha1ctx); > git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); > git_SHA1_Final(hash, &sha1ctx); I think these while-at-it changes should be removed from this commit. Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms 2026-03-04 7:42 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > Nit: we're not exactly deduplicating just yet, but are rather preparing > for that as there is no second implementation using this yet. Fair, renamed the commit to "fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c". > I think these while-at-it changes should be removed from this commit. Done, the rename is now 100% content-identical (the BUG guard and blank line removal are gone). ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 06/10] fsmonitor: deduplicate settings logic for Unix platforms 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (4 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 07/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (5 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The macOS fsm-settings-darwin.c is applicable to other Unix variants as well. Rename it to fsm-settings-unix.c, using the safer xstrdup()+dirname() approach and including the "vfat" filesystem check. Now that both fsm-ipc and fsm-settings use the "unix" variant name, set FSMONITOR_OS_SETTINGS to "unix" for macOS in config.mak.uname and remove the if/else conditionals from the build files. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 6 +---- ...-settings-darwin.c => fsm-settings-unix.c} | 24 +++++++++++------- config.mak.uname | 2 +- contrib/buildsystems/CMakeLists.txt | 25 +++++++++---------- meson.build | 14 +++++------ 5 files changed, 35 insertions(+), 36 deletions(-) rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (82%) diff --git a/Makefile b/Makefile index 7480ce3e1d..062347997a 100644 --- a/Makefile +++ b/Makefile @@ -2365,15 +2365,11 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o -ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o -else - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o -endif endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-unix.c similarity index 82% rename from compat/fsmonitor/fsm-settings-darwin.c rename to compat/fsmonitor/fsm-settings-unix.c index a382590635..27d89207af 100644 --- a/compat/fsmonitor/fsm-settings-darwin.c +++ b/compat/fsmonitor/fsm-settings-unix.c @@ -5,7 +5,7 @@ #include "fsmonitor-settings.h" #include "fsmonitor-path-utils.h" - /* +/* * For the builtin FSMonitor, we create the Unix domain socket for the * IPC in the .git directory. If the working directory is remote, * then the socket will be created on the remote file system. This @@ -22,25 +22,31 @@ * The builtin FSMonitor uses a Unix domain socket in the .git * directory for IPC. These Windows drive formats do not support * Unix domain sockets, so mark them as incompatible for the daemon. - * */ static enum fsmonitor_reason check_uds_volume(struct repository *r) { struct fs_info fs; const char *ipc_path = fsmonitor_ipc__get_path(r); - struct strbuf path = STRBUF_INIT; - strbuf_add(&path, ipc_path, strlen(ipc_path)); + char *path; + char *dir; + + /* + * Create a copy for dirname() since it may modify its argument. + */ + path = xstrdup(ipc_path); + dir = dirname(path); - if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) { - strbuf_release(&path); + if (fsmonitor__get_fs_info(dir, &fs) == -1) { + free(path); return FSMONITOR_REASON_ERROR; } - strbuf_release(&path); + free(path); if (fs.is_remote || - !strcmp(fs.typename, "msdos") || - !strcmp(fs.typename, "ntfs")) { + !strcmp(fs.typename, "msdos") || + !strcmp(fs.typename, "ntfs") || + !strcmp(fs.typename, "vfat")) { free(fs.typename); return FSMONITOR_REASON_NOSOCKETS; } diff --git a/config.mak.uname b/config.mak.uname index 3c35ae33a3..33877020e9 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -165,7 +165,7 @@ ifeq ($(uname_S),Darwin) ifndef NO_PTHREADS ifndef NO_UNIX_SOCKETS FSMONITOR_DAEMON_BACKEND = darwin - FSMONITOR_OS_SETTINGS = darwin + FSMONITOR_OS_SETTINGS = unix endif endif diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 32ef6ebe1b..4099f9a951 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -291,23 +291,22 @@ endif() if(SUPPORTS_SIMPLE_IPC) if(CMAKE_SYSTEM_NAME STREQUAL "Windows") - add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) - - add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) + set(FSMONITOR_DAEMON_BACKEND "win32") + set(FSMONITOR_OS_SETTINGS "win32") elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") + set(FSMONITOR_DAEMON_BACKEND "darwin") + set(FSMONITOR_OS_SETTINGS "unix") + endif() + + if(FSMONITOR_DAEMON_BACKEND) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-${FSMONITOR_OS_SETTINGS}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) endif() endif() diff --git a/meson.build b/meson.build index 8de795f9d4..589624f399 100644 --- a/meson.build +++ b/meson.build @@ -1320,10 +1320,13 @@ else endif fsmonitor_backend = '' +fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' + fsmonitor_os = 'win32' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' + fsmonitor_os = 'unix' libgit_dependencies += dependency('CoreServices') endif if fsmonitor_backend != '' @@ -1334,17 +1337,12 @@ if fsmonitor_backend != '' 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', + 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', ] - - if fsmonitor_backend == 'win32' - libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' - else - libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' - endif endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) +build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_os) if not get_option('b_sanitize').contains('address') and get_option('regex').allowed() and compiler.has_header('regex.h') and compiler.get_define('REG_STARTEND', prefix: '#include <regex.h>') != '' build_options_config.set('NO_REGEX', '') -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 06/10] fsmonitor: deduplicate settings logic for Unix platforms 2026-02-26 0:27 ` [PATCH v7 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget @ 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:43 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:19AM +0000, Paul Tarjan via GitGitGadget wrote: > diff --git a/Makefile b/Makefile > index 7480ce3e1d..062347997a 100644 > --- a/Makefile > +++ b/Makefile > @@ -2365,15 +2365,11 @@ ifdef FSMONITOR_DAEMON_BACKEND > COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND > COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o > COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o > -ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) > - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o > -else > - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o > -endif > endif > > ifdef FSMONITOR_OS_SETTINGS > COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS > + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o > COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o > COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o > endif It's a bit weird that the only change to the Makefile here is to change how we wire up fsm-ipc even though it's fsm-settings that this commit cares about. Should this change be moved into the preceding commit? > diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-unix.c > similarity index 82% > rename from compat/fsmonitor/fsm-settings-darwin.c > rename to compat/fsmonitor/fsm-settings-unix.c > index a382590635..27d89207af 100644 > --- a/compat/fsmonitor/fsm-settings-darwin.c > +++ b/compat/fsmonitor/fsm-settings-unix.c > @@ -5,7 +5,7 @@ > #include "fsmonitor-settings.h" > #include "fsmonitor-path-utils.h" > > - /* > +/* > * For the builtin FSMonitor, we create the Unix domain socket for the > * IPC in the .git directory. If the working directory is remote, > * then the socket will be created on the remote file system. This > @@ -22,25 +22,31 @@ > * The builtin FSMonitor uses a Unix domain socket in the .git > * directory for IPC. These Windows drive formats do not support > * Unix domain sockets, so mark them as incompatible for the daemon. > - * > */ > static enum fsmonitor_reason check_uds_volume(struct repository *r) > { > struct fs_info fs; > const char *ipc_path = fsmonitor_ipc__get_path(r); > - struct strbuf path = STRBUF_INIT; > - strbuf_add(&path, ipc_path, strlen(ipc_path)); > + char *path; > + char *dir; > + > + /* > + * Create a copy for dirname() since it may modify its argument. > + */ > + path = xstrdup(ipc_path); > + dir = dirname(path); > > - if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) { > - strbuf_release(&path); > + if (fsmonitor__get_fs_info(dir, &fs) == -1) { > + free(path); > return FSMONITOR_REASON_ERROR; > } > > - strbuf_release(&path); > + free(path); > > if (fs.is_remote || > - !strcmp(fs.typename, "msdos") || > - !strcmp(fs.typename, "ntfs")) { > + !strcmp(fs.typename, "msdos") || > + !strcmp(fs.typename, "ntfs") || > + !strcmp(fs.typename, "vfat")) { > free(fs.typename); > return FSMONITOR_REASON_NOSOCKETS; > } Same here, it's not clear to me where those while-at-it changes are coming from and whether we need them here. I'd rather drop them. > diff --git a/meson.build b/meson.build > index 8de795f9d4..589624f399 100644 > --- a/meson.build > +++ b/meson.build > @@ -1320,10 +1320,13 @@ else > endif > > fsmonitor_backend = '' > +fsmonitor_os = '' > if host_machine.system() == 'windows' > fsmonitor_backend = 'win32' > + fsmonitor_os = 'win32' > elif host_machine.system() == 'darwin' > fsmonitor_backend = 'darwin' > + fsmonitor_os = 'unix' > libgit_dependencies += dependency('CoreServices') > endif > if fsmonitor_backend != '' I think it might make sense to introduce the `fsmonitor_os` variable in the preceding commit already. If so... > @@ -1334,17 +1337,12 @@ if fsmonitor_backend != '' > 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', > 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', > 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', > - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', > + 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', > + 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', > ] > - > - if fsmonitor_backend == 'win32' > - libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' > - else > - libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' > - endif > endif We could avoid the flip-flopping of the code here. Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 06/10] fsmonitor: deduplicate settings logic for Unix platforms 2026-03-04 7:43 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > It's a bit weird that the only change to the Makefile here is to change > how we wire up fsm-ipc even though it's fsm-settings that this commit > cares about. Should this change be moved into the preceding commit? Moved. The Makefile fsm-ipc change is now in the IPC rename commit (patch 5). The settings commit has no Makefile changes since fsm-settings already used FSMONITOR_OS_SETTINGS. > Same here, it's not clear to me where those while-at-it changes are > coming from and whether we need them here. I'd rather drop them. Dropped. Both renames are now pure renames with no content changes. > I think it might make sense to introduce the `fsmonitor_os` variable in > the preceding commit already. Done, fsmonitor_os is introduced in the IPC rename commit now. > We could avoid the flip-flopping of the code here. Yeah, with fsmonitor_os in the IPC commit, the settings commit just flips fsm-settings from fsmonitor_backend to fsmonitor_os. No more flip-flopping. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 07/10] fsmonitor: implement filesystem change listener for Linux 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (5 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 08/10] fsmonitor: add tests " Paul Tarjan via GitGitGadget ` (4 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS's FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify's cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 220 ++++++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 4 + meson.build | 4 + 8 files changed, 1043 insertions(+), 6 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..36a701c06a --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,746 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <sys/inotify.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + if (errno == ENOSPC) + return error(_("inotify watch limit reached; " + "increase fs.inotify.max_user_watches")); + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->fd_inotify = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd >= 0 && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("Forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + /* + * Interval in seconds between checks for stale directory renames. + * A directory rename that is not completed within this window + * (i.e. no matching IN_MOVED_TO for an IN_MOVED_FROM) indicates + * missed events, forcing a shutdown. + */ + const int interval = 1; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 50); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..b4c19e0655 --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,220 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef V9FS_MAGIC +#define V9FS_MAGIC 0x01021997 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Get the filesystem type name for logging purposes. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case V9FS_MAGIC: + return "9p"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * Returns the filesystem type for the longest matching mount point. + */ +static char *find_mount(const char *path, struct statfs *fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + struct statfs path_fs; + + if (statfs(path, &path_fs) < 0) + return NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + *fs = mount_fs; + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/config.mak.uname b/config.mak.uname index 33877020e9..638f7e1bde 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = unix + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 4099f9a951..00d6e757b6 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -296,6 +296,10 @@ if(SUPPORTS_SIMPLE_IPC) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") set(FSMONITOR_DAEMON_BACKEND "darwin") set(FSMONITOR_OS_SETTINGS "unix") + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + set(FSMONITOR_DAEMON_BACKEND "linux") + set(FSMONITOR_OS_SETTINGS "unix") + add_compile_definitions(HAVE_LINUX_MAGIC_H) endif() if(FSMONITOR_DAEMON_BACKEND) diff --git a/meson.build b/meson.build index 589624f399..0b033a9b9a 100644 --- a/meson.build +++ b/meson.build @@ -1324,6 +1324,10 @@ fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' fsmonitor_os = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + fsmonitor_os = 'unix' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' fsmonitor_os = 'unix' -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 07/10] fsmonitor: implement filesystem change listener for Linux 2026-02-26 0:27 ` [PATCH v7 07/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:43 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:20AM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> > > Implement the built-in fsmonitor daemon for Linux using the inotify > API, bringing it to feature parity with the existing Windows and macOS > implementations. > > The implementation uses inotify rather than fanotify because fanotify > requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it > unsuitable for an unprivileged user-space daemon. While inotify has > the limitation of requiring a separate watch on every directory (unlike > macOS's FSEvents, which can monitor an entire directory tree with a > single watch), it operates without elevated privileges and provides > the per-file event granularity needed for fsmonitor. Thanks for adding this explanation, makes sense. > diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c > new file mode 100644 > index 0000000000..b4c19e0655 > --- /dev/null > +++ b/compat/fsmonitor/fsm-path-utils-linux.c > @@ -0,0 +1,220 @@ > +#include "git-compat-util.h" > +#include "fsmonitor-ll.h" > +#include "fsmonitor-path-utils.h" > +#include "gettext.h" > +#include "trace.h" > + > +#include <sys/statfs.h> > + > +#ifdef HAVE_LINUX_MAGIC_H I saw that this define is only wired up for CMake. I guess we should also add it to our Makefile (probably via config.mak.uname) and Meson (probably via compiler.has_header()). [snip] > +/* > + * Get the filesystem type name for logging purposes. > + */ > +static const char *get_fs_typename(unsigned long f_type) > +{ > + switch (f_type) { > + case CIFS_SUPER_MAGIC: > + return "cifs"; > + case SMB_SUPER_MAGIC: > + return "smb"; > + case SMB2_SUPER_MAGIC: > + return "smb2"; > + case NFS_SUPER_MAGIC: > + return "nfs"; > + case AFS_SUPER_MAGIC: > + return "afs"; > + case CODA_SUPER_MAGIC: > + return "coda"; > + case V9FS_MAGIC: > + return "9p"; > + case FUSE_SUPER_MAGIC: > + return "fuse"; > + default: > + return "unknown"; > + } > +} This selection looks rather interesting to me. Why wouldn't we include common filesystems like ext4 and the like? Certainly hints that the function needs better documentation, and potentially a better name. > +/* > + * Find the mount point for a given path by reading /proc/mounts. > + * Returns the filesystem type for the longest matching mount point. > + */ > +static char *find_mount(const char *path, struct statfs *fs) > +{ > + FILE *fp; > + struct strbuf line = STRBUF_INIT; > + struct strbuf match = STRBUF_INIT; > + struct strbuf fstype = STRBUF_INIT; > + char *result = NULL; > + struct statfs path_fs; > + > + if (statfs(path, &path_fs) < 0) > + return NULL; > + > + fp = fopen("/proc/mounts", "r"); > + if (!fp) > + return NULL; > + > + while (strbuf_getline(&line, fp) != EOF) { > + char *fields[6]; > + char *p = line.buf; > + int i; > + > + /* Parse mount entry: device mountpoint fstype options dump pass */ > + for (i = 0; i < 6 && p; i++) { > + fields[i] = p; > + p = strchr(p, ' '); > + if (p) > + *p++ = '\0'; > + } > + > + if (i >= 3) { > + const char *mountpoint = fields[1]; > + const char *type = fields[2]; > + struct statfs mount_fs; > + > + /* Check if this mount point is a prefix of our path */ > + if (starts_with(path, mountpoint) && > + (path[strlen(mountpoint)] == '/' || > + path[strlen(mountpoint)] == '\0')) { > + /* Check if filesystem ID matches */ > + if (statfs(mountpoint, &mount_fs) == 0 && > + !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, > + sizeof(mount_fs.f_fsid))) { > + /* Keep the longest matching mount point */ > + if (strlen(mountpoint) > match.len) { > + strbuf_reset(&match); > + strbuf_addstr(&match, mountpoint); > + strbuf_reset(&fstype); > + strbuf_addstr(&fstype, type); > + *fs = mount_fs; > + } > + } > + } > + } > + } > + > + fclose(fp); > + strbuf_release(&line); > + strbuf_release(&match); > + > + if (fstype.len) > + result = strbuf_detach(&fstype, NULL); > + else > + strbuf_release(&fstype); > + > + return result; > +} Sorry, but I still don't quite understand what we're doing here. Isn't the longest matching mount point always the one that statfs(3p) gave us? Why do we have to scan "/proc/mounts"? Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 07/10] fsmonitor: implement filesystem change listener for Linux 2026-03-04 7:43 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > I saw that this define is only wired up for CMake. I guess we should > also add it to our Makefile (probably via config.mak.uname) and Meson > (probably via compiler.has_header()). It's already wired up in all three: - config.mak.uname line 78: BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H - meson.build: compiler.has_header('linux/magic.h') check - CMakeLists.txt: add_compile_definitions(HAVE_LINUX_MAGIC_H) While looking at CMakeLists.txt I noticed the GIT-BUILD-OPTIONS string replacements were hardcoded to "win32" for both FSMONITOR_DAEMON_BACKEND and FSMONITOR_OS_SETTINGS. Fixed that to use the CMake variables so Linux/macOS builds get the right values. > This selection looks rather interesting to me. Why wouldn't we include > common filesystems like ext4 and the like? Certainly hints that the > function needs better documentation, and potentially a better name. It's only used as a fallback when /proc/mounts isn't available. Since we only care about naming the remote/special filesystems that is_remote_fs() flags as incompatible, there's no need for ext4 etc. Updated the comment in v8 to make this clearer. Also removed V9FS_MAGIC from this function since 9p is used for local VM/container host mounts where fsmonitor works fine, and it's not in is_remote_fs(). > Sorry, but I still don't quite understand what we're doing here. Isn't > the longest matching mount point always the one that statfs(3p) gave us? > Why do we have to scan "/proc/mounts"? statfs(2) gives us f_type (magic number) and f_fsid but not the human-readable filesystem type string. We need the type name (e.g. "nfs", "cifs") for the fs_info.typename field, which check_uds_volume() compares against strings like "msdos" and "ntfs". /proc/mounts has that string, and we match on f_fsid to find the right entry. Also fixed a redundant statfs() call: find_mount() was calling statfs(path, ...) again even though the caller already had the result. Changed it to take a const pointer to the caller's statfs instead. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 08/10] fsmonitor: add tests for Linux 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 07/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-02-26 0:27 ` [PATCH v7 09/10] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget ` (3 subsequent siblings) 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a smoke test that verifies the filesystem actually delivers inotify events to the daemon. On some configurations (e.g., overlayfs with older kernels), inotify watches succeed but events are never delivered. The daemon cookie wait will time out, but every subsequent test would fail. Skip the entire test file early when this is detected. Add a test that exercises rapid nested directory creation to verify the daemon correctly handles the EEXIST race between recursive scan and queued inotify events. When IN_MASK_CREATE is available and a directory watch is added during recursive registration, the kernel may also deliver a queued IN_CREATE event for the same directory. The second inotify_add_watch() returns EEXIST, which must be treated as harmless. An earlier version of the listener crashed in this scenario. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- t/meson.build | 8 +++- t/t7527-builtin-fsmonitor.sh | 89 +++++++++++++++++++++++++++++++++--- 2 files changed, 89 insertions(+), 8 deletions(-) diff --git a/t/meson.build b/t/meson.build index 19e8306298..85ef2ae2fa 100644 --- a/t/meson.build +++ b/t/meson.build @@ -1210,12 +1210,18 @@ test_environment = script_environment test_environment.set('GIT_BUILD_DIR', git_build_dir) foreach integration_test : integration_tests + per_test_kwargs = test_kwargs + # The fsmonitor tests start daemon processes that in some environments + # can hang. Set a generous timeout to prevent CI from blocking. + if fs.stem(integration_test) == 't7527-builtin-fsmonitor' + per_test_kwargs += {'timeout': 1800} + endif test(fs.stem(integration_test), shell, args: [ integration_test ], workdir: meson.current_source_dir(), env: test_environment, depends: test_dependencies + bin_wrappers, - kwargs: test_kwargs, + kwargs: per_test_kwargs, ) endforeach diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..774da5ac60 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -10,9 +10,58 @@ then test_done fi +# Verify that the filesystem delivers events to the daemon. +# On some configurations (e.g., overlayfs with older kernels), +# inotify watches succeed but events are never delivered. The +# cookie wait will time out and the daemon logs a trace message. +# +# Use "timeout" (if available) to guard each step against hangs. +maybe_timeout () { + if type timeout >/dev/null 2>&1 + then + timeout "$@" + else + shift + "$@" + fi +} +verify_fsmonitor_works () { + git init test_fsmonitor_smoke || return 1 + + GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && + export GIT_TRACE_FSMONITOR && + maybe_timeout 30 \ + git -C test_fsmonitor_smoke fsmonitor--daemon start \ + --start-timeout=10 + ret=$? + unset GIT_TRACE_FSMONITOR + if test $ret -ne 0 + then + rm -rf test_fsmonitor_smoke smoke.trace + return 1 + fi + + maybe_timeout 10 \ + test-tool -C test_fsmonitor_smoke fsmonitor-client query \ + --token 0 >/dev/null 2>&1 + maybe_timeout 5 \ + git -C test_fsmonitor_smoke fsmonitor--daemon stop 2>/dev/null + ! grep -q "cookie_wait timed out" "$PWD/smoke.trace" 2>/dev/null + ret=$? + rm -rf test_fsmonitor_smoke smoke.trace + return $ret +} + +if ! verify_fsmonitor_works +then + skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" + test_done +fi + stop_daemon_delete_repo () { r=$1 && - test_might_fail git -C $r fsmonitor--daemon stop && + test_might_fail maybe_timeout 30 \ + git -C $r fsmonitor--daemon stop 2>/dev/null rm -rf $1 } @@ -67,7 +116,7 @@ start_daemon () { export GIT_TEST_FSMONITOR_TOKEN fi && - git $r fsmonitor--daemon start && + git $r fsmonitor--daemon start --start-timeout=10 && git $r fsmonitor--daemon status ) } @@ -520,6 +569,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the @@ -910,7 +981,10 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" ' start_git_in_background () { git "$@" & git_pid=$! - git_pgid=$(ps -o pgid= -p $git_pid) + git_pgid=$(ps -o pgid= -p $git_pid 2>/dev/null || + awk '{print $5}' /proc/$git_pid/stat 2>/dev/null) && + git_pgid="${git_pgid## }" && + git_pgid="${git_pgid%% }" nr_tries_left=10 while true do @@ -921,15 +995,16 @@ start_git_in_background () { fi sleep 1 nr_tries_left=$(($nr_tries_left - 1)) - done >/dev/null 2>&1 & + done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! wait $git_pid } stop_git () { - while kill -0 -- -$git_pgid + test -n "$git_pgid" || return 0 + while kill -0 -- -$git_pgid 2>/dev/null do - kill -- -$git_pgid + kill -- -$git_pgid 2>/dev/null sleep 1 done } @@ -944,7 +1019,7 @@ stop_watchdog () { test_expect_success !MINGW "submodule implicitly starts daemon by pull" ' test_atexit "stop_watchdog" && - test_when_finished "stop_git; rm -rf cloned super sub" && + test_when_finished "set +m; stop_git; rm -rf cloned super sub" && create_super super && create_sub sub && -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 08/10] fsmonitor: add tests for Linux 2026-02-26 0:27 ` [PATCH v7 08/10] fsmonitor: add tests " Paul Tarjan via GitGitGadget @ 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:43 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:21AM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> > > Add a smoke test that verifies the filesystem actually delivers > inotify events to the daemon. On some configurations (e.g., > overlayfs with older kernels), inotify watches succeed but events > are never delivered. The daemon cookie wait will time out, but > every subsequent test would fail. Skip the entire test file early > when this is detected. Hm. So the fsmonitor listener for Linux is not reliable? Wouldn't the end user see the same issue then? I'm not sure whether just ignoring that issue and adding a timeout to our tests is the proper way to fix it. Before we jump to such solutions I'd rather want to know what the root cause of this. We had similar issues in the past on macOS, where we eventually figured out that we were missing events due to the buffers not being big enough. So did you investigate what the conditions are to trigger this? Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 08/10] fsmonitor: add tests for Linux 2026-03-04 7:43 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > Hm. So the fsmonitor listener for Linux is not reliable? Wouldn't the > end user see the same issue then? I'm not sure whether just ignoring > that issue and adding a timeout to our tests is the proper way to fix > it. > > Before we jump to such solutions I'd rather want to know what the root > cause of this. We had similar issues in the past on macOS, where we > eventually figured out that we were missing events due to the buffers > not being big enough. So did you investigate what the conditions are to > trigger this? This isn't a buffer issue or a bug in our code, it's a kernel limitation in overlayfs. On older kernels, overlayfs doesn't implement the fsnotify hooks that inotify depends on, so inotify_add_watch succeeds but events are silently never delivered. Newer kernels have partially fixed this but it still affects some configurations. An end user on overlayfs would hit the same thing: the cookie wait times out and they get a full-scan fallback. Correct results, just no incremental speedup. The smoke test catches this early so we can skip instead of timing out on every subsequent test. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v7 09/10] run-command: add close_fd_above_stderr option 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (7 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 08/10] fsmonitor: add tests " Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 10/10] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget ` (2 subsequent siblings) 11 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a new option to struct child_process that closes file descriptors 3 and above in the child after forking but before exec. This prevents long-running child processes from inheriting pipe endpoints or other descriptors from the parent environment. The upper bound for the fd scan comes from sysconf(_SC_OPEN_MAX), capped at 4096 to avoid excessive iteration when the limit is set very high. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- run-command.c | 11 +++++++++++ run-command.h | 9 +++++++++ 2 files changed, 20 insertions(+) diff --git a/run-command.c b/run-command.c index e3e02475cc..cbadcf5ff8 100644 --- a/run-command.c +++ b/run-command.c @@ -832,6 +832,17 @@ fail_pipe: child_close(cmd->out); } + if (cmd->close_fd_above_stderr) { + long max_fd = sysconf(_SC_OPEN_MAX); + int fd; + if (max_fd < 0 || max_fd > 4096) + max_fd = 4096; + for (fd = 3; fd < max_fd; fd++) { + if (fd != child_notifier) + close(fd); + } + } + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); diff --git a/run-command.h b/run-command.h index 0df25e445f..a1aa1b1069 100644 --- a/run-command.h +++ b/run-command.h @@ -141,6 +141,15 @@ struct child_process { unsigned stdout_to_stderr:1; unsigned clean_on_exit:1; unsigned wait_after_clean:1; + + /** + * Close file descriptors 3 and above in the child after forking + * but before exec. This prevents the long-running child from + * inheriting pipe endpoints or other descriptors from the parent + * environment (e.g., the test harness). + */ + unsigned close_fd_above_stderr:1; + void (*clean_on_exit_handler)(struct child_process *process); }; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v7 10/10] fsmonitor: close inherited file descriptors and detach in daemon 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (8 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 09/10] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 ` Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-02-26 15:34 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Junio C Hamano 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget 11 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-02-26 0:27 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> When the fsmonitor daemon is spawned as a background process, it may inherit file descriptors from its parent that it does not need. In particular, when the test harness or a CI system captures output through pipes, the daemon can inherit duplicated pipe endpoints. If the daemon holds these open, the parent process never sees EOF and may appear to hang. Set close_fd_above_stderr on the child process at daemon startup so that file descriptors 3 and above are closed before any daemon work begins. This ensures the daemon does not inadvertently hold open descriptors from its launching environment. Additionally, call setsid() when the daemon starts with --detach to create a new session and process group. Without this, shells that enable job control (e.g. bash with "set -m") treat the daemon as part of the spawning command's job. Their "wait" builtin then blocks until the daemon exits, which it never does. This specifically affects systems where /bin/sh is bash (e.g. Fedora), since dash only waits for the specific PID rather than the full process group. Add a 30-second timeout to "fsmonitor--daemon stop" so it does not block indefinitely if the daemon fails to shut down. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 28 +++++++++++++++++++++++++--- fsmonitor-ipc.c | 3 +++ t/meson.build | 8 +------- t/t7527-builtin-fsmonitor.sh | 12 +++++++++++- 4 files changed, 40 insertions(+), 11 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 110fe5fb55..be8f3fee1a 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -86,6 +86,8 @@ static int do_as_client__send_stop(void) { struct strbuf answer = STRBUF_INIT; int ret; + int max_wait_ms = 30000; + int elapsed_ms = 0; ret = fsmonitor_ipc__send_command("quit", &answer); @@ -96,8 +98,16 @@ static int do_as_client__send_stop(void) return ret; trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL); - while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) + while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) { + if (elapsed_ms >= max_wait_ms) { + trace2_region_leave("fsm_client", + "polling-for-daemon-exit", NULL); + return error(_("daemon did not stop within %d seconds"), + max_wait_ms / 1000); + } sleep_millisec(50); + elapsed_ms += 50; + } trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL); return 0; @@ -1431,7 +1441,7 @@ done: return err; } -static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) +static int try_to_run_foreground_daemon(int detach_console) { /* * Technically, we don't need to probe for an existing daemon @@ -1451,10 +1461,21 @@ static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) fflush(stderr); } + if (detach_console) { #ifdef GIT_WINDOWS_NATIVE - if (detach_console) FreeConsole(); +#else + /* + * Create a new session so that the daemon is detached + * from the parent's process group. This prevents + * shells with job control (e.g. bash with "set -m") + * from waiting on the daemon when they wait for a + * foreground command that implicitly spawned it. + */ + if (setsid() == -1) + warning_errno(_("setsid failed")); #endif + } return !!fsmonitor_run_daemon(); } @@ -1517,6 +1538,7 @@ static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; + cp.close_fd_above_stderr = 1; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c index f1b1631111..6112d13064 100644 --- a/fsmonitor-ipc.c +++ b/fsmonitor-ipc.c @@ -61,6 +61,9 @@ static int spawn_daemon(void) cmd.git_cmd = 1; cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; + cmd.close_fd_above_stderr = 1; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); diff --git a/t/meson.build b/t/meson.build index 85ef2ae2fa..19e8306298 100644 --- a/t/meson.build +++ b/t/meson.build @@ -1210,18 +1210,12 @@ test_environment = script_environment test_environment.set('GIT_BUILD_DIR', git_build_dir) foreach integration_test : integration_tests - per_test_kwargs = test_kwargs - # The fsmonitor tests start daemon processes that in some environments - # can hang. Set a generous timeout to prevent CI from blocking. - if fs.stem(integration_test) == 't7527-builtin-fsmonitor' - per_test_kwargs += {'timeout': 1800} - endif test(fs.stem(integration_test), shell, args: [ integration_test ], workdir: meson.current_source_dir(), env: test_environment, depends: test_dependencies + bin_wrappers, - kwargs: per_test_kwargs, + kwargs: test_kwargs, ) endforeach diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 774da5ac60..d7e64bcb7a 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -766,7 +766,7 @@ do else test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] enable fsmonitor" ' git config core.fsmonitor true && - git fsmonitor--daemon start && + git fsmonitor--daemon start --start-timeout=10 && git update-index --fsmonitor ' fi @@ -997,7 +997,17 @@ start_git_in_background () { nr_tries_left=$(($nr_tries_left - 1)) done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! + + # Disable job control before wait. With "set -m", bash treats + # "wait $pid" as waiting for the entire job (process group), + # which blocks indefinitely if the fsmonitor daemon was spawned + # into the same process group and is still running. Turning off + # job control makes "wait" only wait for the specific PID. + set +m && wait $git_pid + wait_status=$? + set -m + return $wait_status } stop_git () { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v7 10/10] fsmonitor: close inherited file descriptors and detach in daemon 2026-02-26 0:27 ` [PATCH v7 10/10] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget @ 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-04 7:43 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan On Thu, Feb 26, 2026 at 12:27:23AM +0000, Paul Tarjan via GitGitGadget wrote: > From: Paul Tarjan <github@paulisageek.com> > > When the fsmonitor daemon is spawned as a background process, it may > inherit file descriptors from its parent that it does not need. In > particular, when the test harness or a CI system captures output through > pipes, the daemon can inherit duplicated pipe endpoints. If the daemon > holds these open, the parent process never sees EOF and may appear to > hang. Oh, interesting. > Set close_fd_above_stderr on the child process at daemon startup so > that file descriptors 3 and above are closed before any daemon work > begins. This ensures the daemon does not inadvertently hold open > descriptors from its launching environment. > > Additionally, call setsid() when the daemon starts with --detach to > create a new session and process group. Without this, shells that > enable job control (e.g. bash with "set -m") treat the daemon as part > of the spawning command's job. Their "wait" builtin then blocks until > the daemon exits, which it never does. This specifically affects > systems where /bin/sh is bash (e.g. Fedora), since dash only waits for > the specific PID rather than the full process group. Hm. We already have related logic in `daemonize()`. Should we maybe reuse that function, and potentially expand it to handle closing all FDs up to the maximum file descriptor? > Add a 30-second timeout to "fsmonitor--daemon stop" so it does > not block indefinitely if the daemon fails to shut down. This feels like a "while-at-it" change to me. Should it maybe be moved into a separate commit? > diff --git a/t/meson.build b/t/meson.build > index 85ef2ae2fa..19e8306298 100644 > --- a/t/meson.build > +++ b/t/meson.build > @@ -1210,18 +1210,12 @@ test_environment = script_environment > test_environment.set('GIT_BUILD_DIR', git_build_dir) > > foreach integration_test : integration_tests > - per_test_kwargs = test_kwargs > - # The fsmonitor tests start daemon processes that in some environments > - # can hang. Set a generous timeout to prevent CI from blocking. > - if fs.stem(integration_test) == 't7527-builtin-fsmonitor' > - per_test_kwargs += {'timeout': 1800} > - endif > test(fs.stem(integration_test), shell, > args: [ integration_test ], > workdir: meson.current_source_dir(), > env: test_environment, > depends: test_dependencies + bin_wrappers, > - kwargs: per_test_kwargs, > + kwargs: test_kwargs, > ) > endforeach > Might make sense to reorder commits a bit so that the fix comes first. In that case we wouldn't ever have to introduce the timeouts in the first place. > diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh > index 774da5ac60..d7e64bcb7a 100755 > --- a/t/t7527-builtin-fsmonitor.sh > +++ b/t/t7527-builtin-fsmonitor.sh > @@ -766,7 +766,7 @@ do > else > test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] enable fsmonitor" ' > git config core.fsmonitor true && > - git fsmonitor--daemon start && > + git fsmonitor--daemon start --start-timeout=10 && > git update-index --fsmonitor > ' > fi This change feels unrelated and is not mentioned in the commit message. > @@ -997,7 +997,17 @@ start_git_in_background () { > nr_tries_left=$(($nr_tries_left - 1)) > done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & > watchdog_pid=$! > + > + # Disable job control before wait. With "set -m", bash treats > + # "wait $pid" as waiting for the entire job (process group), > + # which blocks indefinitely if the fsmonitor daemon was spawned > + # into the same process group and is still running. Turning off > + # job control makes "wait" only wait for the specific PID. > + set +m && > wait $git_pid > + wait_status=$? > + set -m > + return $wait_status > } I thought with our call to setsid() we're not part of the same process group anymore. So why is this change here still needed? Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 10/10] fsmonitor: close inherited file descriptors and detach in daemon 2026-03-04 7:43 ` Patrick Steinhardt @ 2026-03-04 18:17 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-04 18:17 UTC (permalink / raw) To: git; +Cc: ps, paul On Tue, Mar 4, 2026, Patrick Steinhardt wrote: > Hm. We already have related logic in `daemonize()`. Should we maybe > reuse that function, and potentially expand it to handle closing all FDs > up to the maximum file descriptor? daemonize() does a double-fork with setsid, which is the classic Unix daemon pattern. But fsmonitor uses start_bg_command(), which forks the daemon and waits for it to signal readiness over IPC. If we called daemonize() inside the child, start_bg_command() would lose track of the PID. So instead we just use setsid() to detach from the terminal, and close_fd_above_stderr in run-command to close inherited FDs before exec. > This feels like a "while-at-it" change to me. Should it maybe be moved > into a separate commit? Done. Split the 30-second stop timeout into its own commit (patch 10, "fsmonitor: add timeout to daemon stop command"). > Might make sense to reorder commits a bit so that the fix comes first. > In that case we wouldn't ever have to introduce the timeouts in the > first place. Done. Reordered in v8 so run-command and daemon detach come before the test commit. The meson timeout never appears now. > This change feels unrelated and is not mentioned in the commit message. --start-timeout=10 is now in the tests commit (patch 11) and documented in that commit message. > I thought with our call to setsid() we're not part of the same process > group anymore. So why is this change here still needed? setsid() runs inside the daemon after it's already been forked. The set -m in the test is about the shell putting `git pull &` into its own process group. Without it, the background job inherits the test shell's pgid and `wait` stalls. Moved to the tests commit (patch 11) with a note in the commit message explaining why it's still needed. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (9 preceding siblings ...) 2026-02-26 0:27 ` [PATCH v7 10/10] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget @ 2026-02-26 15:34 ` Junio C Hamano 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget 11 siblings, 0 replies; 129+ messages in thread From: Junio C Hamano @ 2026-02-26 15:34 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Patrick Steinhardt, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > Changes since v6: > > * Introduced FSMONITOR_OS_SETTINGS build variable (set to "unix" for macOS > and Linux, "win32" for Windows) to eliminate if/else conditionals in > Makefile, meson.build, and CMakeLists.txt per Junio's review > * Moved fsm-path-utils from FSMONITOR_OS_SETTINGS to > FSMONITOR_DAEMON_BACKEND since path-utils files are platform-specific > * Removed V9FS_MAGIC from remote filesystem detection (9p is used for local > VM/container host mounts where fsmonitor works fine) > * Removed redundant #include <libgen.h> (already provided by > compat/posix.h) > * Fixed cookie wait comment wording ("see" → "observe") > * Rewrote commit messages for IPC and settings dedup patches I saw nothing unexpected in this iteration relative to the previous one. Looking good. It would be nice to have another set of eyes on the inotify part of this series before we mark it for 'next'. Thanks. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v8 00/12] fsmonitor: implement filesystem change listener for Linux 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (10 preceding siblings ...) 2026-02-26 15:34 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Junio C Hamano @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget ` (12 more replies) 11 siblings, 13 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan This series implements the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. It also fixes two memory leaks in the platform-independent daemon code and deduplicates the IPC and settings logic that is now shared between macOS and Linux. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. This series builds on work from https://github.com/git/git/pull/1352 by Eric DeCosta and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v7: * Added patch 12: convert khash to strset in do_handle_client (Patrick's #leftoverbit suggestion) * Fixed "Forcing shutdown" trace message to start with lowercase * Fixed redundant statfs() call in find_mount() (caller already had the result) * Fixed CMakeLists.txt GIT-BUILD-OPTIONS: was hardcoded to "win32" for FSMONITOR_DAEMON_BACKEND and FSMONITOR_OS_SETTINGS, now uses the CMake variables * Fixed uninitialized strset on trivial response path (STRSET_INIT) * Removed V9FS_MAGIC from get_fs_typename() to match is_remote_fs() (9p is local VM mounts) * Split 30-second stop timeout into its own commit per review request * Fixed misleading indentation on shutdown assignment in handle_events() * Updated commit messages to describe all changes (test hardening, fsmonitor-ipc.c spawn changes) * Updated Makefile comment for FSMONITOR_OS_SETTINGS to mention fsm-ipc Changes since v6: * Introduced FSMONITOR_OS_SETTINGS build variable (set to "unix" for macOS and Linux, "win32" for Windows) to eliminate if/else conditionals in Makefile, meson.build, and CMakeLists.txt per Junio's review * Moved fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND since path-utils files are platform-specific * Removed V9FS_MAGIC from remote filesystem detection (9p is used for local VM/container host mounts where fsmonitor works fine) * Removed redundant #include <libgen.h> (already provided by compat/posix.h) * Fixed cookie wait comment wording ("see" to "observe") * Rewrote commit messages for IPC and settings dedup patches Changes since v5: * Split monolithic commit into 10-patch series per Patrick's review * Deduplicated fsm-ipc and fsm-settings into shared Unix implementations * Rewrote commit message with prose paragraphs, explain inotify vs fanotify, removed "Issues addressed" sections, added Based-on-patch-by trailers * Removed redundant includes already provided by compat/posix.h * Fixed error/trace message capitalization per coding guidelines * Fixed stale rename check interval from 1000 seconds to 1 second * Changed poll timeout from 1ms to 50ms to reduce idle CPU wake-ups * Replaced infinite pthread_cond_wait cookie loop with one-second pthread_cond_timedwait (prevents daemon hangs on overlay filesystems where events are never delivered) * Added pthread_cond_timedwait to Windows pthread compatibility layer * Separated test into its own commit with smoke test that skips when inotify events are not delivered (e.g., overlayfs with older kernels) * Fixed test hang on Fedora CI: stop_git() looped forever when ps was unavailable because bash in POSIX/sh mode returns exit 0 from kill with an empty process group argument. Fixed by falling back to /proc/$pid/stat for process group ID and guarding stop_git against empty pgid. * Redirect spawn_daemon() stdout/stderr to /dev/null and close inherited file descriptors to prevent the intermediate process from holding test pipe file descriptors * Call setsid() on daemon detach to prevent shells with job control from waiting on the daemon process group * Close inherited file descriptors 3-7 in the test watchdog subprocess * Added 30-second timeout to "fsmonitor--daemon stop" to prevent indefinite blocking * Added helpful error message when inotify watch limit (max_user_watches) is reached * Initialize fd_inotify to -1 and use fd >= 0 check for correct fd 0 handling * Use sysconf(_SC_OPEN_MAX) instead of hardcoded 1024 for fd close limit * Check setsid() return value Changes since v4: * Added Meson build support Changes since v3: * Fix crash on rapid nested directory creation (EEXIST from inotify_add_watch with IN_MASK_CREATE) * Extensive stress testing Changes since v2: * Fix khash memory leak in do_handle_client Changes since v1: * Fix hashmap memory leak in fsmonitor_run_daemon() Paul Tarjan (12): fsmonitor: fix khash memory leak in do_handle_client fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon compat/win32: add pthread_cond_timedwait fsmonitor: use pthread_cond_timedwait for cookie wait fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c fsmonitor: implement filesystem change listener for Linux run-command: add close_fd_above_stderr option fsmonitor: close inherited file descriptors and detach in daemon fsmonitor: add timeout to daemon stop command fsmonitor: add tests for Linux fsmonitor: convert shown khash to strset in do_handle_client Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- Makefile | 6 +- builtin/fsmonitor--daemon.c | 92 ++- compat/fsmonitor/fsm-health-linux.c | 33 + .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 +++++ ...-settings-darwin.c => fsm-settings-unix.c} | 0 compat/win32/pthread.c | 26 + compat/win32/pthread.h | 2 + config.mak.uname | 12 +- contrib/buildsystems/CMakeLists.txt | 33 +- fsmonitor-ipc.c | 3 + meson.build | 13 +- run-command.c | 11 + run-command.h | 9 + t/t7527-builtin-fsmonitor.sh | 89 ++- 18 files changed, 1261 insertions(+), 63 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) base-commit: 3e0db84c88c57e70ac8be8c196dfa92c5d656fbc Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v8 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v8 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v7: 1: 4d4dec8fa1 = 1: 4d4dec8fa1 fsmonitor: fix khash memory leak in do_handle_client 2: d0bd3e32ca ! 2: cb270120f0 fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon @@ Commit message The `state.cookies` hashmap is initialized during daemon startup but never freed during cleanup in the `done:` label of - fsmonitor_run_daemon(). Add a hashmap_clear() call to prevent this - memory leak. + fsmonitor_run_daemon(). The cookie entries also have names allocated + via strbuf_detach() that must be freed individually. + + Iterate the hashmap to free each cookie name, then call + hashmap_clear_and_free() to release the entries and table. Signed-off-by: Paul Tarjan <github@paulisageek.com> @@ builtin/fsmonitor--daemon.c: static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); -+ hashmap_clear(&state.cookies); ++ { ++ struct hashmap_iter iter; ++ struct fsmonitor_cookie_item *cookie; ++ ++ hashmap_for_each_entry(&state.cookies, &iter, cookie, entry) ++ free(cookie->name); ++ hashmap_clear_and_free(&state.cookies, ++ struct fsmonitor_cookie_item, entry); ++ } fsm_listen__dtor(&state); fsm_health__dtor(&state); 3: d2c5ca0939 ! 3: 44a063074d compat/win32: add pthread_cond_timedwait @@ Commit message compatibility layer using SleepConditionVariableCS() with a millisecond timeout computed from the absolute deadline. - This enables callers to use bounded waits on condition variables - instead of blocking indefinitely with pthread_cond_wait(). - Signed-off-by: Paul Tarjan <github@paulisageek.com> ## compat/win32/pthread.c ## @@ compat/win32/pthread.c: int pthread_cond_wait(pthread_cond_t *cond, pthread_mute + abstime->tv_nsec / 1000000; + + if (deadline_ms <= now_ms) -+ timeout_ms = 0; ++ return ETIMEDOUT; + else + timeout_ms = (DWORD)(deadline_ms - now_ms); + 4: 0a58670952 = 4: b1081d1e13 fsmonitor: use pthread_cond_timedwait for cookie wait 6: 0a83bb9c8e ! 5: dec0fb144f fsmonitor: deduplicate settings logic for Unix platforms @@ Metadata Author: Paul Tarjan <github@paulisageek.com> ## Commit message ## - fsmonitor: deduplicate settings logic for Unix platforms + fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c - The macOS fsm-settings-darwin.c is applicable to other Unix variants - as well. Rename it to fsm-settings-unix.c, using the safer - xstrdup()+dirname() approach and including the "vfat" filesystem check. + The fsmonitor IPC path logic in fsm-ipc-darwin.c is not + Darwin-specific and will be reused by the upcoming Linux + implementation. Rename it to fsm-ipc-unix.c to reflect that it + is shared by all Unix platforms. - Now that both fsm-ipc and fsm-settings use the "unix" variant name, - set FSMONITOR_OS_SETTINGS to "unix" for macOS in config.mak.uname and - remove the if/else conditionals from the build files. + Introduce FSMONITOR_OS_SETTINGS (set to "unix" for non-Windows, "win32" + for Windows) as a separate variable from FSMONITOR_DAEMON_BACKEND so + that the build files can distinguish between platform-specific files + (listen, health, path-utils) and shared Unix files (ipc, settings). + Move fsm-ipc to the FSMONITOR_OS_SETTINGS section in the Makefile, and + switch fsm-path-utils to use FSMONITOR_DAEMON_BACKEND since path-utils + is platform-specific (there will be separate darwin and linux versions). + + Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> + Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> ## Makefile ## +@@ Makefile: include shared.mak + # If your platform has OS-specific ways to tell if a repo is incompatible with + # fsmonitor (whether the hook or IPC daemon version), set FSMONITOR_OS_SETTINGS + # to the "<name>" of the corresponding `compat/fsmonitor/fsm-settings-<name>.c` +-# that implements the `fsm_os_settings__*()` routines. ++# and `compat/fsmonitor/fsm-ipc-<name>.c` files. + # + # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test + # programs in oss-fuzz/. @@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o --ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) -- COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o --else -- COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o --endif +- COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o - COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o +- COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o ++ COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif - - ## compat/fsmonitor/fsm-settings-darwin.c => compat/fsmonitor/fsm-settings-unix.c ## -@@ - #include "fsmonitor-settings.h" - #include "fsmonitor-path-utils.h" - -- /* -+/* - * For the builtin FSMonitor, we create the Unix domain socket for the - * IPC in the .git directory. If the working directory is remote, - * then the socket will be created on the remote file system. This -@@ - * The builtin FSMonitor uses a Unix domain socket in the .git - * directory for IPC. These Windows drive formats do not support - * Unix domain sockets, so mark them as incompatible for the daemon. -- * - */ - static enum fsmonitor_reason check_uds_volume(struct repository *r) - { - struct fs_info fs; - const char *ipc_path = fsmonitor_ipc__get_path(r); -- struct strbuf path = STRBUF_INIT; -- strbuf_add(&path, ipc_path, strlen(ipc_path)); -+ char *path; -+ char *dir; -+ -+ /* -+ * Create a copy for dirname() since it may modify its argument. -+ */ -+ path = xstrdup(ipc_path); -+ dir = dirname(path); -- if (fsmonitor__get_fs_info(dirname(path.buf), &fs) == -1) { -- strbuf_release(&path); -+ if (fsmonitor__get_fs_info(dir, &fs) == -1) { -+ free(path); - return FSMONITOR_REASON_ERROR; - } - -- strbuf_release(&path); -+ free(path); - - if (fs.is_remote || -- !strcmp(fs.typename, "msdos") || -- !strcmp(fs.typename, "ntfs")) { -+ !strcmp(fs.typename, "msdos") || -+ !strcmp(fs.typename, "ntfs") || -+ !strcmp(fs.typename, "vfat")) { - free(fs.typename); - return FSMONITOR_REASON_NOSOCKETS; - } + ifdef WITH_BREAKING_CHANGES + + ## compat/fsmonitor/fsm-ipc-darwin.c => compat/fsmonitor/fsm-ipc-unix.c ## ## config.mak.uname ## @@ config.mak.uname: ifeq ($(uname_S),Darwin) @@ contrib/buildsystems/CMakeLists.txt: endif() add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) -- list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-${FSMONITOR_DAEMON_BACKEND}.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-${FSMONITOR_OS_SETTINGS}.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) endif() endif() @@ meson.build: else endif if fsmonitor_backend != '' @@ meson.build: if fsmonitor_backend != '' + + libgit_sources += [ 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', +- 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', ++ 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', -- 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', -+ 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', -+ 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', + 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] -- -- if fsmonitor_backend == 'win32' -- libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' -- else -- libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' -- endif endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) 5: 037ae2a03f ! 6: b2aaadb4ae fsmonitor: deduplicate IPC path logic for Unix platforms @@ Metadata Author: Paul Tarjan <github@paulisageek.com> ## Commit message ## - fsmonitor: deduplicate IPC path logic for Unix platforms + fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c - The macOS fsm-ipc-darwin.c is applicable to other Unix variants as - well. Rename it to fsm-ipc-unix.c and add a worktree NULL check - (BUG guard) that was missing from the macOS version. + The fsmonitor settings logic in fsm-settings-darwin.c is not + Darwin-specific and will be reused by the upcoming Linux + implementation. Rename it to fsm-settings-unix.c to reflect that it + is shared by all Unix platforms. - To support this, introduce FSMONITOR_OS_SETTINGS which is set to - "unix" for both macOS and Linux, distinct from FSMONITOR_DAEMON_BACKEND - which remains platform-specific (darwin, linux, win32). Move - fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND - since the path-utils files are platform-specific. + Update the build files (meson.build and CMakeLists.txt) to use + FSMONITOR_OS_SETTINGS for fsm-settings, matching the approach already + used for fsm-ipc. + Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> + Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> - ## Makefile ## -@@ Makefile: ifdef FSMONITOR_DAEMON_BACKEND - COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND - COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o - COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o -- COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o -+ifeq ($(FSMONITOR_DAEMON_BACKEND),win32) -+ COMPAT_OBJS += compat/fsmonitor/fsm-ipc-win32.o -+else -+ COMPAT_OBJS += compat/fsmonitor/fsm-ipc-unix.o -+endif - endif - - ifdef FSMONITOR_OS_SETTINGS - COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS - COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o -- COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o -+ COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o - endif - - ifdef WITH_BREAKING_CHANGES - - ## compat/fsmonitor/fsm-ipc-darwin.c => compat/fsmonitor/fsm-ipc-unix.c ## -@@ compat/fsmonitor/fsm-ipc-unix.c: const char *fsmonitor_ipc__get_path(struct repository *r) - if (ipc_path) - return ipc_path; - -- - /* By default the socket file is created in the .git directory */ - if (fsmonitor__is_fs_remote(r->gitdir) < 1) { - ipc_path = fsmonitor_ipc__get_default_path(); - return ipc_path; - } - -+ if (!r->worktree) -+ BUG("repository has no worktree"); -+ - git_SHA1_Init(&sha1ctx); - git_SHA1_Update(&sha1ctx, r->worktree, strlen(r->worktree)); - git_SHA1_Final(hash, &sha1ctx); + ## compat/fsmonitor/fsm-settings-darwin.c => compat/fsmonitor/fsm-settings-unix.c ## ## contrib/buildsystems/CMakeLists.txt ## @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) - add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) -- list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) -+ list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-unix.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) +- list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) ++ list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) + endif() + endif() + ## meson.build ## @@ meson.build: if fsmonitor_backend != '' - - libgit_sources += [ - 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', -- 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', +- 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ++ 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', ] -+ -+ if fsmonitor_backend == 'win32' -+ libgit_sources += 'compat/fsmonitor/fsm-ipc-win32.c' -+ else -+ libgit_sources += 'compat/fsmonitor/fsm-ipc-unix.c' -+ endif endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) - build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) 7: adac7964cc ! 7: 03cf12d01b fsmonitor: implement filesystem change listener for Linux @@ compat/fsmonitor/fsm-listen-linux.c (new) + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); -+ state->listen_data->shutdown = SHUTDOWN_ERROR; ++ state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + @@ compat/fsmonitor/fsm-listen-linux.c (new) + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) -+ log_mask_set("Forcing shutdown", event->mask); ++ log_mask_set("forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } @@ compat/fsmonitor/fsm-path-utils-linux.c (new) +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif -+#ifndef V9FS_MAGIC -+#define V9FS_MAGIC 0x01021997 -+#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif @@ compat/fsmonitor/fsm-path-utils-linux.c (new) +} + +/* -+ * Get the filesystem type name for logging purposes. ++ * Map filesystem magic numbers to human-readable names as a fallback ++ * when /proc/mounts is unavailable. This only covers the remote and ++ * special filesystems in is_remote_fs() above; local filesystems are ++ * never flagged as incompatible, so we do not need their names here. + */ +static const char *get_fs_typename(unsigned long f_type) +{ @@ compat/fsmonitor/fsm-path-utils-linux.c (new) + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; -+ case V9FS_MAGIC: -+ return "9p"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: @@ compat/fsmonitor/fsm-path-utils-linux.c (new) + +/* + * Find the mount point for a given path by reading /proc/mounts. -+ * Returns the filesystem type for the longest matching mount point. ++ * ++ * statfs(2) gives us f_type (the magic number) but not the human-readable ++ * filesystem type string. We scan /proc/mounts to find the mount entry ++ * whose path is the longest prefix of ours and whose f_fsid matches, ++ * which gives us the fstype string (e.g. "nfs", "ext4") for logging. + */ -+static char *find_mount(const char *path, struct statfs *fs) ++static char *find_mount(const char *path, const struct statfs *path_fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; -+ struct statfs path_fs; -+ -+ if (statfs(path, &path_fs) < 0) -+ return NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) @@ compat/fsmonitor/fsm-path-utils-linux.c (new) + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && -+ !memcmp(&mount_fs.f_fsid, &path_fs.f_fsid, ++ !memcmp(&mount_fs.f_fsid, &path_fs->f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { @@ compat/fsmonitor/fsm-path-utils-linux.c (new) + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); -+ *fs = mount_fs; + } + } + } @@ contrib/buildsystems/CMakeLists.txt: if(SUPPORTS_SIMPLE_IPC) endif() if(FSMONITOR_DAEMON_BACKEND) +@@ contrib/buildsystems/CMakeLists.txt: endif() + file(STRINGS ${CMAKE_SOURCE_DIR}/GIT-BUILD-OPTIONS.in git_build_options NEWLINE_CONSUME) + string(REPLACE "@BROKEN_PATH_FIX@" "" git_build_options "${git_build_options}") + string(REPLACE "@DIFF@" "'${DIFF}'" git_build_options "${git_build_options}") +-string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "win32" git_build_options "${git_build_options}") +-string(REPLACE "@FSMONITOR_OS_SETTINGS@" "win32" git_build_options "${git_build_options}") ++string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "${FSMONITOR_DAEMON_BACKEND}" git_build_options "${git_build_options}") ++string(REPLACE "@FSMONITOR_OS_SETTINGS@" "${FSMONITOR_OS_SETTINGS}" git_build_options "${git_build_options}") + string(REPLACE "@GITWEBDIR@" "'${GITWEBDIR}'" git_build_options "${git_build_options}") + string(REPLACE "@GIT_INTEROP_MAKE_OPTS@" "" git_build_options "${git_build_options}") + string(REPLACE "@GIT_PERF_LARGE_REPO@" "" git_build_options "${git_build_options}") ## meson.build ## @@ meson.build: fsmonitor_os = '' 9: c684fc9094 ! 8: 29a6461915 run-command: add close_fd_above_stderr option @@ Commit message run-command: add close_fd_above_stderr option Add a new option to struct child_process that closes file descriptors - 3 and above in the child after forking but before exec. This prevents - long-running child processes from inheriting pipe endpoints or other + 3 and above in the child after forking but before exec. Without this, + long-running child processes inherit pipe endpoints and other descriptors from the parent environment. The upper bound for the fd scan comes from sysconf(_SC_OPEN_MAX), 10: 4987a009a2 ! 9: b596c5004d fsmonitor: close inherited file descriptors and detach in daemon @@ Commit message holds these open, the parent process never sees EOF and may appear to hang. - Set close_fd_above_stderr on the child process at daemon startup so - that file descriptors 3 and above are closed before any daemon work - begins. This ensures the daemon does not inadvertently hold open - descriptors from its launching environment. + Set close_fd_above_stderr on the child process at both daemon startup + paths: the explicit "fsmonitor--daemon start" command and the implicit + spawn triggered by fsmonitor-ipc when a client finds no running daemon. + Also suppress stdout and stderr on the implicit spawn path to prevent + the background daemon from writing to the client's terminal. Additionally, call setsid() when the daemon starts with --detach to - create a new session and process group. Without this, shells that - enable job control (e.g. bash with "set -m") treat the daemon as part - of the spawning command's job. Their "wait" builtin then blocks until - the daemon exits, which it never does. This specifically affects - systems where /bin/sh is bash (e.g. Fedora), since dash only waits for - the specific PID rather than the full process group. - - Add a 30-second timeout to "fsmonitor--daemon stop" so it does - not block indefinitely if the daemon fails to shut down. + create a new session and process group. This prevents the daemon + from being part of the spawning shell's process group, which could + cause the shell's "wait" to block until the daemon exits. Signed-off-by: Paul Tarjan <github@paulisageek.com> ## builtin/fsmonitor--daemon.c ## -@@ builtin/fsmonitor--daemon.c: static int do_as_client__send_stop(void) - { - struct strbuf answer = STRBUF_INIT; - int ret; -+ int max_wait_ms = 30000; -+ int elapsed_ms = 0; - - ret = fsmonitor_ipc__send_command("quit", &answer); - -@@ builtin/fsmonitor--daemon.c: static int do_as_client__send_stop(void) - return ret; - - trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL); -- while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) -+ while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) { -+ if (elapsed_ms >= max_wait_ms) { -+ trace2_region_leave("fsm_client", -+ "polling-for-daemon-exit", NULL); -+ return error(_("daemon did not stop within %d seconds"), -+ max_wait_ms / 1000); -+ } - sleep_millisec(50); -+ elapsed_ms += 50; -+ } - trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL); - - return 0; @@ builtin/fsmonitor--daemon.c: done: return err; } @@ fsmonitor-ipc.c: static int spawn_daemon(void) cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); - - ## t/meson.build ## -@@ t/meson.build: test_environment = script_environment - test_environment.set('GIT_BUILD_DIR', git_build_dir) - - foreach integration_test : integration_tests -- per_test_kwargs = test_kwargs -- # The fsmonitor tests start daemon processes that in some environments -- # can hang. Set a generous timeout to prevent CI from blocking. -- if fs.stem(integration_test) == 't7527-builtin-fsmonitor' -- per_test_kwargs += {'timeout': 1800} -- endif - test(fs.stem(integration_test), shell, - args: [ integration_test ], - workdir: meson.current_source_dir(), - env: test_environment, - depends: test_dependencies + bin_wrappers, -- kwargs: per_test_kwargs, -+ kwargs: test_kwargs, - ) - endforeach - - - ## t/t7527-builtin-fsmonitor.sh ## -@@ t/t7527-builtin-fsmonitor.sh: do - else - test_expect_success "Matrix[uc:$uc_val][fsm:$fsm_val] enable fsmonitor" ' - git config core.fsmonitor true && -- git fsmonitor--daemon start && -+ git fsmonitor--daemon start --start-timeout=10 && - git update-index --fsmonitor - ' - fi -@@ t/t7527-builtin-fsmonitor.sh: start_git_in_background () { - nr_tries_left=$(($nr_tries_left - 1)) - done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & - watchdog_pid=$! -+ -+ # Disable job control before wait. With "set -m", bash treats -+ # "wait $pid" as waiting for the entire job (process group), -+ # which blocks indefinitely if the fsmonitor daemon was spawned -+ # into the same process group and is still running. Turning off -+ # job control makes "wait" only wait for the specific PID. -+ set +m && - wait $git_pid -+ wait_status=$? -+ set -m -+ return $wait_status - } - - stop_git () { -: ---------- > 10: 72125ac20f fsmonitor: add timeout to daemon stop command 8: fad2f0a81a ! 11: 76b171ab5c fsmonitor: add tests for Linux @@ Commit message as harmless. An earlier version of the listener crashed in this scenario. - Signed-off-by: Paul Tarjan <github@paulisageek.com> + Reduce --start-timeout from the default 60 seconds to 10 seconds so + that tests fail promptly when the daemon cannot start. - ## t/meson.build ## -@@ t/meson.build: test_environment = script_environment - test_environment.set('GIT_BUILD_DIR', git_build_dir) - - foreach integration_test : integration_tests -+ per_test_kwargs = test_kwargs -+ # The fsmonitor tests start daemon processes that in some environments -+ # can hang. Set a generous timeout to prevent CI from blocking. -+ if fs.stem(integration_test) == 't7527-builtin-fsmonitor' -+ per_test_kwargs += {'timeout': 1800} -+ endif - test(fs.stem(integration_test), shell, - args: [ integration_test ], - workdir: meson.current_source_dir(), - env: test_environment, - depends: test_dependencies + bin_wrappers, -- kwargs: test_kwargs, -+ kwargs: per_test_kwargs, - ) - endforeach - + Harden the test helpers to work in environments without procps + (e.g., Fedora CI): fall back to reading /proc/$pid/stat for the + process group ID when ps is unavailable, guard stop_git() against + an empty pgid, and redirect stderr from kill to /dev/null to avoid + noise when processes have already exited. + + Use set -m to enable job control in the submodule-pull test so that + the background git pull gets its own process group, preventing the + shell wait from blocking on the daemon. setsid() in the previous + commit detaches the daemon itself, but the intermediate git pull + process still needs its own process group for the test shell to + manage it correctly. + + Signed-off-by: Paul Tarjan <github@paulisageek.com> ## t/t7527-builtin-fsmonitor.sh ## @@ t/t7527-builtin-fsmonitor.sh: then -: ---------- > 12: 6c36c9e11e fsmonitor: convert shown khash to strset in do_handle_client -- gitgitgadget ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v8 01/12] fsmonitor: fix khash memory leak in do_handle_client 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget ` (11 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `shown` kh_str_t was freed with kh_release_str() at a point in the code only reachable in the non-trivial response path. When the client receives a trivial response, the code jumps to the `cleanup` label, skipping the kh_release_str() call entirely and leaking the hash table. Fix this by initializing `shown` to NULL and moving the cleanup to the `cleanup` label using kh_destroy_str(), which is safe to call on NULL. This ensures the hash table is freed regardless of which code path is taken. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..bc4571938c 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget ` (10 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `state.cookies` hashmap is initialized during daemon startup but never freed during cleanup in the `done:` label of fsmonitor_run_daemon(). The cookie entries also have names allocated via strbuf_detach() that must be freed individually. Iterate the hashmap to free each cookie name, then call hashmap_clear_and_free() to release the entries and table. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index bc4571938c..d8d32b01ef 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1404,6 +1404,15 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + { + struct hashmap_iter iter; + struct fsmonitor_cookie_item *cookie; + + hashmap_for_each_entry(&state.cookies, &iter, cookie, entry) + free(cookie->name); + hashmap_clear_and_free(&state.cookies, + struct fsmonitor_cookie_item, entry); + } fsm_listen__dtor(&state); fsm_health__dtor(&state); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 03/12] compat/win32: add pthread_cond_timedwait 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget ` (9 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pthread_cond_timedwait() implementation to the Windows pthread compatibility layer using SleepConditionVariableCS() with a millisecond timeout computed from the absolute deadline. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/win32/pthread.c | 26 ++++++++++++++++++++++++++ compat/win32/pthread.h | 2 ++ 2 files changed, 28 insertions(+) diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c index 7e93146963..398caa9602 100644 --- a/compat/win32/pthread.c +++ b/compat/win32/pthread.c @@ -66,3 +66,29 @@ int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) return err_win_to_posix(GetLastError()); return 0; } + +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime) +{ + struct timeval now; + long long now_ms, deadline_ms; + DWORD timeout_ms; + + gettimeofday(&now, NULL); + now_ms = (long long)now.tv_sec * 1000 + now.tv_usec / 1000; + deadline_ms = (long long)abstime->tv_sec * 1000 + + abstime->tv_nsec / 1000000; + + if (deadline_ms <= now_ms) + return ETIMEDOUT; + else + timeout_ms = (DWORD)(deadline_ms - now_ms); + + if (SleepConditionVariableCS(cond, mutex, timeout_ms) == 0) { + DWORD err = GetLastError(); + if (err == ERROR_TIMEOUT) + return ETIMEDOUT; + return err_win_to_posix(err); + } + return 0; +} diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index ccacc5a53b..d80df8d12a 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -64,6 +64,8 @@ int win32_pthread_join(pthread_t *thread, void **value_ptr); pthread_t pthread_self(void); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime); static inline void NORETURN pthread_exit(void *ret) { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (2 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget ` (8 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The cookie wait in with_lock__wait_for_cookie() uses an infinite pthread_cond_wait() loop. The existing comment notes the desire to switch to pthread_cond_timedwait(), but the routine was not available in git thread-utils. On certain container or overlay filesystems, inotify watches may succeed but events are never delivered. In this case the daemon would hang indefinitely waiting for the cookie event, which in turn causes the client to hang. Replace the infinite wait with a one-second timeout using pthread_cond_timedwait(). If the timeout fires, report an error and let the client proceed with a trivial (full-scan) response rather than blocking forever. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index d8d32b01ef..c8ec7b722e 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -197,20 +197,31 @@ static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie( unlink(cookie_pathname.buf); /* - * Technically, this is an infinite wait (well, unless another - * thread sends us an abort). I'd like to change this to - * use `pthread_cond_timedwait()` and return an error/timeout - * and let the caller do the trivial response thing, but we - * don't have that routine in our thread-utils. - * - * After extensive beta testing I'm not really worried about - * this. Also note that the above open() and unlink() calls - * will cause at least two FS events on that path, so the odds - * of getting stuck are pretty slim. + * Wait for the listener thread to observe the cookie file. + * Time out after a short interval so that the client + * does not hang forever if the filesystem does not deliver + * events (e.g., on certain container/overlay filesystems + * where inotify watches succeed but events never arrive). */ - while (cookie->result == FCIR_INIT) - pthread_cond_wait(&state->cookies_cond, - &state->main_lock); + { + struct timeval now; + struct timespec ts; + int err = 0; + + gettimeofday(&now, NULL); + ts.tv_sec = now.tv_sec + 1; + ts.tv_nsec = now.tv_usec * 1000; + + while (cookie->result == FCIR_INIT && !err) + err = pthread_cond_timedwait(&state->cookies_cond, + &state->main_lock, + &ts); + if (err == ETIMEDOUT && cookie->result == FCIR_INIT) { + trace_printf_key(&trace_fsmonitor, + "cookie_wait timed out"); + cookie->result = FCIR_ERROR; + } + } done: hashmap_remove(&state->cookies, &cookie->entry, NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (3 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget ` (7 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor IPC path logic in fsm-ipc-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-ipc-unix.c to reflect that it is shared by all Unix platforms. Introduce FSMONITOR_OS_SETTINGS (set to "unix" for non-Windows, "win32" for Windows) as a separate variable from FSMONITOR_DAEMON_BACKEND so that the build files can distinguish between platform-specific files (listen, health, path-utils) and shared Unix files (ipc, settings). Move fsm-ipc to the FSMONITOR_OS_SETTINGS section in the Makefile, and switch fsm-path-utils to use FSMONITOR_DAEMON_BACKEND since path-utils is platform-specific (there will be separate darwin and linux versions). Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 6 ++--- .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 config.mak.uname | 2 +- contrib/buildsystems/CMakeLists.txt | 25 +++++++++---------- meson.build | 7 ++++-- 5 files changed, 21 insertions(+), 19 deletions(-) rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) diff --git a/Makefile b/Makefile index 8aa489f3b6..080d009bf0 100644 --- a/Makefile +++ b/Makefile @@ -417,7 +417,7 @@ include shared.mak # If your platform has OS-specific ways to tell if a repo is incompatible with # fsmonitor (whether the hook or IPC daemon version), set FSMONITOR_OS_SETTINGS # to the "<name>" of the corresponding `compat/fsmonitor/fsm-settings-<name>.c` -# that implements the `fsm_os_settings__*()` routines. +# and `compat/fsmonitor/fsm-ipc-<name>.c` files. # # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test # programs in oss-fuzz/. @@ -2365,13 +2365,13 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o - COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef WITH_BREAKING_CHANGES diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-unix.c similarity index 100% rename from compat/fsmonitor/fsm-ipc-darwin.c rename to compat/fsmonitor/fsm-ipc-unix.c diff --git a/config.mak.uname b/config.mak.uname index 3c35ae33a3..33877020e9 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -165,7 +165,7 @@ ifeq ($(uname_S),Darwin) ifndef NO_PTHREADS ifndef NO_UNIX_SOCKETS FSMONITOR_DAEMON_BACKEND = darwin - FSMONITOR_OS_SETTINGS = darwin + FSMONITOR_OS_SETTINGS = unix endif endif diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..6197d5729c 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -291,23 +291,22 @@ endif() if(SUPPORTS_SIMPLE_IPC) if(CMAKE_SYSTEM_NAME STREQUAL "Windows") - add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) - - add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) + set(FSMONITOR_DAEMON_BACKEND "win32") + set(FSMONITOR_OS_SETTINGS "win32") elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") + set(FSMONITOR_DAEMON_BACKEND "darwin") + set(FSMONITOR_OS_SETTINGS "unix") + endif() + + if(FSMONITOR_DAEMON_BACKEND) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-${FSMONITOR_OS_SETTINGS}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) endif() endif() diff --git a/meson.build b/meson.build index dd52efd1c8..86a68365a9 100644 --- a/meson.build +++ b/meson.build @@ -1320,10 +1320,13 @@ else endif fsmonitor_backend = '' +fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' + fsmonitor_os = 'win32' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' + fsmonitor_os = 'unix' libgit_dependencies += dependency('CoreServices') endif if fsmonitor_backend != '' @@ -1332,14 +1335,14 @@ if fsmonitor_backend != '' libgit_sources += [ 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) +build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_os) if not get_option('b_sanitize').contains('address') and get_option('regex').allowed() and compiler.has_header('regex.h') and compiler.get_define('REG_STARTEND', prefix: '#include <regex.h>') != '' build_options_config.set('NO_REGEX', '') -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (4 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor settings logic in fsm-settings-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-settings-unix.c to reflect that it is shared by all Unix platforms. Update the build files (meson.build and CMakeLists.txt) to use FSMONITOR_OS_SETTINGS for fsm-settings, matching the approach already used for fsm-ipc. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} | 0 contrib/buildsystems/CMakeLists.txt | 2 +- meson.build | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-unix.c similarity index 100% rename from compat/fsmonitor/fsm-settings-darwin.c rename to compat/fsmonitor/fsm-settings-unix.c diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 6197d5729c..d613809e26 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -306,7 +306,7 @@ if(SUPPORTS_SIMPLE_IPC) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) endif() endif() diff --git a/meson.build b/meson.build index 86a68365a9..4f0c0a33b8 100644 --- a/meson.build +++ b/meson.build @@ -1338,7 +1338,7 @@ if fsmonitor_backend != '' 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 07/12] fsmonitor: implement filesystem change listener for Linux 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (5 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 08/12] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget ` (5 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS's FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify's cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 ++++++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 8 +- meson.build | 4 + 8 files changed, 1042 insertions(+), 8 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..e3dca14b62 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,746 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <sys/inotify.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + if (errno == ENOSPC) + return error(_("inotify watch limit reached; " + "increase fs.inotify.max_user_watches")); + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->fd_inotify = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd >= 0 && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + /* + * Interval in seconds between checks for stale directory renames. + * A directory rename that is not completed within this window + * (i.e. no matching IN_MOVED_TO for an IN_MOVED_FROM) indicates + * missed events, forcing a shutdown. + */ + const int interval = 1; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 50); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..c9866b1b24 --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,217 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Map filesystem magic numbers to human-readable names as a fallback + * when /proc/mounts is unavailable. This only covers the remote and + * special filesystems in is_remote_fs() above; local filesystems are + * never flagged as incompatible, so we do not need their names here. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * + * statfs(2) gives us f_type (the magic number) but not the human-readable + * filesystem type string. We scan /proc/mounts to find the mount entry + * whose path is the longest prefix of ours and whose f_fsid matches, + * which gives us the fstype string (e.g. "nfs", "ext4") for logging. + */ +static char *find_mount(const char *path, const struct statfs *path_fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs->f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/config.mak.uname b/config.mak.uname index 33877020e9..638f7e1bde 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = unix + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index d613809e26..b7da108f29 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -296,6 +296,10 @@ if(SUPPORTS_SIMPLE_IPC) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") set(FSMONITOR_DAEMON_BACKEND "darwin") set(FSMONITOR_OS_SETTINGS "unix") + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + set(FSMONITOR_DAEMON_BACKEND "linux") + set(FSMONITOR_OS_SETTINGS "unix") + add_compile_definitions(HAVE_LINUX_MAGIC_H) endif() if(FSMONITOR_DAEMON_BACKEND) @@ -1149,8 +1153,8 @@ endif() file(STRINGS ${CMAKE_SOURCE_DIR}/GIT-BUILD-OPTIONS.in git_build_options NEWLINE_CONSUME) string(REPLACE "@BROKEN_PATH_FIX@" "" git_build_options "${git_build_options}") string(REPLACE "@DIFF@" "'${DIFF}'" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "win32" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_OS_SETTINGS@" "win32" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "${FSMONITOR_DAEMON_BACKEND}" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_OS_SETTINGS@" "${FSMONITOR_OS_SETTINGS}" git_build_options "${git_build_options}") string(REPLACE "@GITWEBDIR@" "'${GITWEBDIR}'" git_build_options "${git_build_options}") string(REPLACE "@GIT_INTEROP_MAKE_OPTS@" "" git_build_options "${git_build_options}") string(REPLACE "@GIT_PERF_LARGE_REPO@" "" git_build_options "${git_build_options}") diff --git a/meson.build b/meson.build index 4f0c0a33b8..123d218460 100644 --- a/meson.build +++ b/meson.build @@ -1324,6 +1324,10 @@ fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' fsmonitor_os = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + fsmonitor_os = 'unix' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' fsmonitor_os = 'unix' -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 08/12] run-command: add close_fd_above_stderr option 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (6 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 20:51 ` Junio C Hamano 2026-03-04 18:15 ` [PATCH v8 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget ` (4 subsequent siblings) 12 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a new option to struct child_process that closes file descriptors 3 and above in the child after forking but before exec. Without this, long-running child processes inherit pipe endpoints and other descriptors from the parent environment. The upper bound for the fd scan comes from sysconf(_SC_OPEN_MAX), capped at 4096 to avoid excessive iteration when the limit is set very high. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- run-command.c | 11 +++++++++++ run-command.h | 9 +++++++++ 2 files changed, 20 insertions(+) diff --git a/run-command.c b/run-command.c index e3e02475cc..cbadcf5ff8 100644 --- a/run-command.c +++ b/run-command.c @@ -832,6 +832,17 @@ fail_pipe: child_close(cmd->out); } + if (cmd->close_fd_above_stderr) { + long max_fd = sysconf(_SC_OPEN_MAX); + int fd; + if (max_fd < 0 || max_fd > 4096) + max_fd = 4096; + for (fd = 3; fd < max_fd; fd++) { + if (fd != child_notifier) + close(fd); + } + } + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); diff --git a/run-command.h b/run-command.h index 0df25e445f..a1aa1b1069 100644 --- a/run-command.h +++ b/run-command.h @@ -141,6 +141,15 @@ struct child_process { unsigned stdout_to_stderr:1; unsigned clean_on_exit:1; unsigned wait_after_clean:1; + + /** + * Close file descriptors 3 and above in the child after forking + * but before exec. This prevents the long-running child from + * inheriting pipe endpoints or other descriptors from the parent + * environment (e.g., the test harness). + */ + unsigned close_fd_above_stderr:1; + void (*clean_on_exit_handler)(struct child_process *process); }; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v8 08/12] run-command: add close_fd_above_stderr option 2026-03-04 18:15 ` [PATCH v8 08/12] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget @ 2026-03-04 20:51 ` Junio C Hamano 2026-03-05 0:49 ` [PATCH v8 09/12] " Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-03-04 20:51 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget Cc: git, Patrick Steinhardt, Paul Tarjan, Paul Tarjan "Paul Tarjan via GitGitGadget" <gitgitgadget@gmail.com> writes: > From: Paul Tarjan <github@paulisageek.com> > > Add a new option to struct child_process that closes file descriptors > 3 and above in the child after forking but before exec. Without this, > long-running child processes inherit pipe endpoints and other > descriptors from the parent environment. > > The upper bound for the fd scan comes from sysconf(_SC_OPEN_MAX), > capped at 4096 to avoid excessive iteration when the limit is set > very high. > > Signed-off-by: Paul Tarjan <github@paulisageek.com> > --- > run-command.c | 11 +++++++++++ > run-command.h | 9 +++++++++ > 2 files changed, 20 insertions(+) > > diff --git a/run-command.c b/run-command.c > index e3e02475cc..cbadcf5ff8 100644 > --- a/run-command.c > +++ b/run-command.c > @@ -832,6 +832,17 @@ fail_pipe: > child_close(cmd->out); > } > > + if (cmd->close_fd_above_stderr) { > + long max_fd = sysconf(_SC_OPEN_MAX); > + int fd; > + if (max_fd < 0 || max_fd > 4096) > + max_fd = 4096; > + for (fd = 3; fd < max_fd; fd++) { > + if (fd != child_notifier) > + close(fd); > + } > + } > + > if (cmd->dir && chdir(cmd->dir)) > child_die(CHILD_ERR_CHDIR); The need for this particular "close file descriptors other than the standard ones" may be common enough that I do not mind to have it inside "run-command.c", but I wonder if a generic callback function to call here in the child between fork and exec that the caller can supply would be a good thing to have. Then, any caller who may want to set close_fd_above_stderr could instead prepare a callback that does the body of the above if statement themselves. > diff --git a/run-command.h b/run-command.h > index 0df25e445f..a1aa1b1069 100644 > --- a/run-command.h > +++ b/run-command.h > @@ -141,6 +141,15 @@ struct child_process { > unsigned stdout_to_stderr:1; > unsigned clean_on_exit:1; > unsigned wait_after_clean:1; > + > + /** > + * Close file descriptors 3 and above in the child after forking > + * but before exec. This prevents the long-running child from > + * inheriting pipe endpoints or other descriptors from the parent > + * environment (e.g., the test harness). > + */ > + unsigned close_fd_above_stderr:1; > + > void (*clean_on_exit_handler)(struct child_process *process); > }; ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v8 09/12] run-command: add close_fd_above_stderr option 2026-03-04 20:51 ` Junio C Hamano @ 2026-03-05 0:49 ` Paul Tarjan 2026-03-05 4:13 ` Junio C Hamano 0 siblings, 1 reply; 129+ messages in thread From: Paul Tarjan @ 2026-03-05 0:49 UTC (permalink / raw) To: git; +Cc: gitster, ps, paul On Wed, Mar 4, 2026, Junio C Hamano wrote: > I wonder if a generic callback function > to call here in the child between fork and exec that the caller can > supply would be a good thing to have. Done in v9. Replaced the close_fd_above_stderr flag with a pre_exec_cb function pointer on struct child_process. The fd-closing logic is now a standalone close_fd_above_stderr() function that the two fsmonitor callers pass as the callback. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v8 09/12] run-command: add close_fd_above_stderr option 2026-03-05 0:49 ` [PATCH v8 09/12] " Paul Tarjan @ 2026-03-05 4:13 ` Junio C Hamano 2026-03-05 6:38 ` [PATCH v9 09/12] run-command: add pre-exec callback for child processes Paul Tarjan 0 siblings, 1 reply; 129+ messages in thread From: Junio C Hamano @ 2026-03-05 4:13 UTC (permalink / raw) To: Paul Tarjan; +Cc: git, ps Paul Tarjan <paul@paultarjan.com> writes: > On Wed, Mar 4, 2026, Junio C Hamano wrote: >> I wonder if a generic callback function >> to call here in the child between fork and exec that the caller can >> supply would be a good thing to have. > > Done in v9. Replaced the close_fd_above_stderr flag with a pre_exec_cb > function pointer on struct child_process. The fd-closing logic is now > a standalone close_fd_above_stderr() function that the two fsmonitor > callers pass as the callback. I didn't mean to suggest using a generic mechanism to _replace_ what you added. A truly generic callback mechanism that will be useful can and should wait until we see real use cases for one. And I strongly suspect that the callback would want to take some callback data argument, not "void cb(void)", but more like "int cb(void *)" (we may find a return value that lets us tell the run_command() to abort instead of exec(2)ingg, for example---and we can make a better design when we do have real use cases. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v9 09/12] run-command: add pre-exec callback for child processes 2026-03-05 4:13 ` Junio C Hamano @ 2026-03-05 6:38 ` Paul Tarjan 0 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-05 6:38 UTC (permalink / raw) To: gitster; +Cc: git, ps, paul, Paul Tarjan Junio C Hamano <gitster@pobox.com> writes: > I didn't mean to suggest using a generic mechanism to _replace_ what > you added. > > A truly generic callback mechanism that will be useful can and > should wait until we see real use cases for one. Haha, got it. I'll put it back and let you mull it over for another patch. ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v8 09/12] fsmonitor: close inherited file descriptors and detach in daemon 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (7 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 08/12] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget ` (3 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> When the fsmonitor daemon is spawned as a background process, it may inherit file descriptors from its parent that it does not need. In particular, when the test harness or a CI system captures output through pipes, the daemon can inherit duplicated pipe endpoints. If the daemon holds these open, the parent process never sees EOF and may appear to hang. Set close_fd_above_stderr on the child process at both daemon startup paths: the explicit "fsmonitor--daemon start" command and the implicit spawn triggered by fsmonitor-ipc when a client finds no running daemon. Also suppress stdout and stderr on the implicit spawn path to prevent the background daemon from writing to the client's terminal. Additionally, call setsid() when the daemon starts with --detach to create a new session and process group. This prevents the daemon from being part of the spawning shell's process group, which could cause the shell's "wait" to block until the daemon exits. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 16 ++++++++++++++-- fsmonitor-ipc.c | 3 +++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index c8ec7b722e..b2a816dc3f 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1439,7 +1439,7 @@ done: return err; } -static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) +static int try_to_run_foreground_daemon(int detach_console) { /* * Technically, we don't need to probe for an existing daemon @@ -1459,10 +1459,21 @@ static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) fflush(stderr); } + if (detach_console) { #ifdef GIT_WINDOWS_NATIVE - if (detach_console) FreeConsole(); +#else + /* + * Create a new session so that the daemon is detached + * from the parent's process group. This prevents + * shells with job control (e.g. bash with "set -m") + * from waiting on the daemon when they wait for a + * foreground command that implicitly spawned it. + */ + if (setsid() == -1) + warning_errno(_("setsid failed")); #endif + } return !!fsmonitor_run_daemon(); } @@ -1525,6 +1536,7 @@ static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; + cp.close_fd_above_stderr = 1; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c index f1b1631111..6112d13064 100644 --- a/fsmonitor-ipc.c +++ b/fsmonitor-ipc.c @@ -61,6 +61,9 @@ static int spawn_daemon(void) cmd.git_cmd = 1; cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; + cmd.close_fd_above_stderr = 1; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 10/12] fsmonitor: add timeout to daemon stop command 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (8 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget ` (2 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The "fsmonitor--daemon stop" command polls in a loop waiting for the daemon to exit after sending a "quit" command over IPC. If the daemon fails to shut down (e.g. it is stuck or wedged), this loop spins forever. Add a 30-second timeout so the stop command returns an error instead of blocking indefinitely. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index b2a816dc3f..53d8ad1f0d 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -86,6 +86,8 @@ static int do_as_client__send_stop(void) { struct strbuf answer = STRBUF_INIT; int ret; + int max_wait_ms = 30000; + int elapsed_ms = 0; ret = fsmonitor_ipc__send_command("quit", &answer); @@ -96,8 +98,16 @@ static int do_as_client__send_stop(void) return ret; trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL); - while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) + while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) { + if (elapsed_ms >= max_wait_ms) { + trace2_region_leave("fsm_client", + "polling-for-daemon-exit", NULL); + return error(_("daemon did not stop within %d seconds"), + max_wait_ms / 1000); + } sleep_millisec(50); + elapsed_ms += 50; + } trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL); return 0; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 11/12] fsmonitor: add tests for Linux 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (9 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a smoke test that verifies the filesystem actually delivers inotify events to the daemon. On some configurations (e.g., overlayfs with older kernels), inotify watches succeed but events are never delivered. The daemon cookie wait will time out, but every subsequent test would fail. Skip the entire test file early when this is detected. Add a test that exercises rapid nested directory creation to verify the daemon correctly handles the EEXIST race between recursive scan and queued inotify events. When IN_MASK_CREATE is available and a directory watch is added during recursive registration, the kernel may also deliver a queued IN_CREATE event for the same directory. The second inotify_add_watch() returns EEXIST, which must be treated as harmless. An earlier version of the listener crashed in this scenario. Reduce --start-timeout from the default 60 seconds to 10 seconds so that tests fail promptly when the daemon cannot start. Harden the test helpers to work in environments without procps (e.g., Fedora CI): fall back to reading /proc/$pid/stat for the process group ID when ps is unavailable, guard stop_git() against an empty pgid, and redirect stderr from kill to /dev/null to avoid noise when processes have already exited. Use set -m to enable job control in the submodule-pull test so that the background git pull gets its own process group, preventing the shell wait from blocking on the daemon. setsid() in the previous commit detaches the daemon itself, but the intermediate git pull process still needs its own process group for the test shell to manage it correctly. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- t/t7527-builtin-fsmonitor.sh | 89 +++++++++++++++++++++++++++++++++--- 1 file changed, 82 insertions(+), 7 deletions(-) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..774da5ac60 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -10,9 +10,58 @@ then test_done fi +# Verify that the filesystem delivers events to the daemon. +# On some configurations (e.g., overlayfs with older kernels), +# inotify watches succeed but events are never delivered. The +# cookie wait will time out and the daemon logs a trace message. +# +# Use "timeout" (if available) to guard each step against hangs. +maybe_timeout () { + if type timeout >/dev/null 2>&1 + then + timeout "$@" + else + shift + "$@" + fi +} +verify_fsmonitor_works () { + git init test_fsmonitor_smoke || return 1 + + GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && + export GIT_TRACE_FSMONITOR && + maybe_timeout 30 \ + git -C test_fsmonitor_smoke fsmonitor--daemon start \ + --start-timeout=10 + ret=$? + unset GIT_TRACE_FSMONITOR + if test $ret -ne 0 + then + rm -rf test_fsmonitor_smoke smoke.trace + return 1 + fi + + maybe_timeout 10 \ + test-tool -C test_fsmonitor_smoke fsmonitor-client query \ + --token 0 >/dev/null 2>&1 + maybe_timeout 5 \ + git -C test_fsmonitor_smoke fsmonitor--daemon stop 2>/dev/null + ! grep -q "cookie_wait timed out" "$PWD/smoke.trace" 2>/dev/null + ret=$? + rm -rf test_fsmonitor_smoke smoke.trace + return $ret +} + +if ! verify_fsmonitor_works +then + skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" + test_done +fi + stop_daemon_delete_repo () { r=$1 && - test_might_fail git -C $r fsmonitor--daemon stop && + test_might_fail maybe_timeout 30 \ + git -C $r fsmonitor--daemon stop 2>/dev/null rm -rf $1 } @@ -67,7 +116,7 @@ start_daemon () { export GIT_TEST_FSMONITOR_TOKEN fi && - git $r fsmonitor--daemon start && + git $r fsmonitor--daemon start --start-timeout=10 && git $r fsmonitor--daemon status ) } @@ -520,6 +569,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the @@ -910,7 +981,10 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" ' start_git_in_background () { git "$@" & git_pid=$! - git_pgid=$(ps -o pgid= -p $git_pid) + git_pgid=$(ps -o pgid= -p $git_pid 2>/dev/null || + awk '{print $5}' /proc/$git_pid/stat 2>/dev/null) && + git_pgid="${git_pgid## }" && + git_pgid="${git_pgid%% }" nr_tries_left=10 while true do @@ -921,15 +995,16 @@ start_git_in_background () { fi sleep 1 nr_tries_left=$(($nr_tries_left - 1)) - done >/dev/null 2>&1 & + done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! wait $git_pid } stop_git () { - while kill -0 -- -$git_pgid + test -n "$git_pgid" || return 0 + while kill -0 -- -$git_pgid 2>/dev/null do - kill -- -$git_pgid + kill -- -$git_pgid 2>/dev/null sleep 1 done } @@ -944,7 +1019,7 @@ stop_watchdog () { test_expect_success !MINGW "submodule implicitly starts daemon by pull" ' test_atexit "stop_watchdog" && - test_when_finished "stop_git; rm -rf cloned super sub" && + test_when_finished "set +m; stop_git; rm -rf cloned super sub" && create_super super && create_sub sub && -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v8 12/12] fsmonitor: convert shown khash to strset in do_handle_client 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (10 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-04 18:15 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Replace the khash-based string set used for deduplicating pathnames in do_handle_client() with a strset, which provides a cleaner interface for the same purpose. Since the paths are interned strings from the batch data, use strdup_strings=0 to avoid unnecessary copies. Suggested-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 53d8ad1f0d..f920cf3a82 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -16,7 +16,7 @@ #include "fsmonitor--daemon.h" #include "simple-ipc.h" -#include "khash.h" +#include "strmap.h" #include "run-command.h" #include "trace.h" #include "trace2.h" @@ -674,8 +674,6 @@ static int fsmonitor_parse_client_token(const char *buf_token, return 0; } -KHASH_INIT(str, const char *, int, 0, kh_str_hash_func, kh_str_hash_equal) - static int do_handle_client(struct fsmonitor_daemon_state *state, const char *command, ipc_server_reply_cb *reply, @@ -692,8 +690,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown = NULL; - int hash_ret; + struct strset shown = STRSET_INIT; int do_trivial = 0; int do_flush = 0; int do_cookie = 0; @@ -882,14 +879,14 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, * so walk the batch list backwards from the current head back * to the batch (sequence number) they named. * - * We use khash to de-dup the list of pathnames. + * We use a strset to de-dup the list of pathnames. * * NEEDSWORK: each batch contains a list of interned strings, * so we only need to do pointer comparisons here to build the * hash table. Currently, we're still comparing the string * values. */ - shown = kh_init_str(); + strset_init_with_options(&shown, NULL, 0); for (batch = batch_head; batch && batch->batch_seq_nr > requested_oldest_seq_nr; batch = batch->next) { @@ -899,11 +896,9 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const char *s = batch->interned_paths[k]; size_t s_len; - if (kh_get_str(shown, s) != kh_end(shown)) + if (!strset_add(&shown, s)) duplicates++; else { - kh_put_str(shown, s, &hash_ret); - trace_printf_key(&trace_fsmonitor, "send[%"PRIuMAX"]: %s", count, s); @@ -973,7 +968,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: - kh_destroy_str(shown); + strset_clear(&shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget ` (11 preceding siblings ...) 2026-03-04 18:15 ` [PATCH v8 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget ` (12 more replies) 12 siblings, 13 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan This series implements the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. It also fixes two memory leaks in the platform-independent daemon code and deduplicates the IPC and settings logic that is now shared between macOS and Linux. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. This series builds on work from https://github.com/git/git/pull/1352 by Eric DeCosta and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v8: * Replaced close_fd_above_stderr flag with generic pre_exec_cb callback in struct child_process per Junio's review * Provided close_fd_above_stderr() as a ready-made callback function Changes since v7: * Added patch 12: convert khash to strset in do_handle_client (Patrick's #leftoverbit suggestion) * Fixed "Forcing shutdown" trace message to start with lowercase * Fixed redundant statfs() call in find_mount() (caller already had the result) * Fixed CMakeLists.txt GIT-BUILD-OPTIONS: was hardcoded to "win32" for FSMONITOR_DAEMON_BACKEND and FSMONITOR_OS_SETTINGS, now uses the CMake variables * Fixed uninitialized strset on trivial response path (STRSET_INIT) * Removed V9FS_MAGIC from get_fs_typename() to match is_remote_fs() (9p is local VM mounts) * Split 30-second stop timeout into its own commit per review request * Fixed misleading indentation on shutdown assignment in handle_events() * Updated commit messages to describe all changes (test hardening, fsmonitor-ipc.c spawn changes) * Updated Makefile comment for FSMONITOR_OS_SETTINGS to mention fsm-ipc Changes since v6: * Introduced FSMONITOR_OS_SETTINGS build variable (set to "unix" for macOS and Linux, "win32" for Windows) to eliminate if/else conditionals in Makefile, meson.build, and CMakeLists.txt per Junio's review * Moved fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND since path-utils files are platform-specific * Removed V9FS_MAGIC from remote filesystem detection (9p is used for local VM/container host mounts where fsmonitor works fine) * Removed redundant #include <libgen.h> (already provided by compat/posix.h) * Fixed cookie wait comment wording ("see" to "observe") * Rewrote commit messages for IPC and settings dedup patches Changes since v5: * Split monolithic commit into 10-patch series per Patrick's review * Deduplicated fsm-ipc and fsm-settings into shared Unix implementations * Rewrote commit message with prose paragraphs, explain inotify vs fanotify, removed "Issues addressed" sections, added Based-on-patch-by trailers * Removed redundant includes already provided by compat/posix.h * Fixed error/trace message capitalization per coding guidelines * Fixed stale rename check interval from 1000 seconds to 1 second * Changed poll timeout from 1ms to 50ms to reduce idle CPU wake-ups * Replaced infinite pthread_cond_wait cookie loop with one-second pthread_cond_timedwait (prevents daemon hangs on overlay filesystems where events are never delivered) * Added pthread_cond_timedwait to Windows pthread compatibility layer * Separated test into its own commit with smoke test that skips when inotify events are not delivered (e.g., overlayfs with older kernels) * Fixed test hang on Fedora CI: stop_git() looped forever when ps was unavailable because bash in POSIX/sh mode returns exit 0 from kill with an empty process group argument. Fixed by falling back to /proc/$pid/stat for process group ID and guarding stop_git against empty pgid. * Redirect spawn_daemon() stdout/stderr to /dev/null and close inherited file descriptors to prevent the intermediate process from holding test pipe file descriptors * Call setsid() on daemon detach to prevent shells with job control from waiting on the daemon process group * Close inherited file descriptors 3-7 in the test watchdog subprocess * Added 30-second timeout to "fsmonitor--daemon stop" to prevent indefinite blocking * Added helpful error message when inotify watch limit (max_user_watches) is reached * Initialize fd_inotify to -1 and use fd >= 0 check for correct fd 0 handling * Use sysconf(_SC_OPEN_MAX) instead of hardcoded 1024 for fd close limit * Check setsid() return value Changes since v4: * Added Meson build support Changes since v3: * Fix crash on rapid nested directory creation (EEXIST from inotify_add_watch with IN_MASK_CREATE) * Extensive stress testing Changes since v2: * Fix khash memory leak in do_handle_client Changes since v1: * Fix hashmap memory leak in fsmonitor_run_daemon() Paul Tarjan (12): fsmonitor: fix khash memory leak in do_handle_client fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon compat/win32: add pthread_cond_timedwait fsmonitor: use pthread_cond_timedwait for cookie wait fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c fsmonitor: implement filesystem change listener for Linux run-command: add pre-exec callback for child processes fsmonitor: close inherited file descriptors and detach in daemon fsmonitor: add timeout to daemon stop command fsmonitor: add tests for Linux fsmonitor: convert shown khash to strset in do_handle_client Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- Makefile | 6 +- builtin/fsmonitor--daemon.c | 92 ++- compat/fsmonitor/fsm-health-linux.c | 33 + .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 +++++ ...-settings-darwin.c => fsm-settings-unix.c} | 0 compat/win32/pthread.c | 26 + compat/win32/pthread.h | 2 + config.mak.uname | 12 +- contrib/buildsystems/CMakeLists.txt | 33 +- fsmonitor-ipc.c | 3 + meson.build | 13 +- run-command.c | 15 + run-command.h | 15 + t/t7527-builtin-fsmonitor.sh | 89 ++- 18 files changed, 1271 insertions(+), 63 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) base-commit: 3e0db84c88c57e70ac8be8c196dfa92c5d656fbc Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v9 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v9 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v8: 1: 4d4dec8fa1 = 1: 4d4dec8fa1 fsmonitor: fix khash memory leak in do_handle_client 2: cb270120f0 = 2: cb270120f0 fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 3: 44a063074d = 3: 44a063074d compat/win32: add pthread_cond_timedwait 4: b1081d1e13 = 4: b1081d1e13 fsmonitor: use pthread_cond_timedwait for cookie wait 5: dec0fb144f = 5: dec0fb144f fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c 6: b2aaadb4ae = 6: b2aaadb4ae fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c 7: 03cf12d01b = 7: 03cf12d01b fsmonitor: implement filesystem change listener for Linux 8: 29a6461915 ! 8: 31fa1fb324 run-command: add close_fd_above_stderr option @@ Metadata Author: Paul Tarjan <github@paulisageek.com> ## Commit message ## - run-command: add close_fd_above_stderr option + run-command: add pre-exec callback for child processes - Add a new option to struct child_process that closes file descriptors - 3 and above in the child after forking but before exec. Without this, - long-running child processes inherit pipe endpoints and other - descriptors from the parent environment. + Add a pre_exec_cb function pointer to struct child_process that is + invoked in the child between fork and exec. This gives callers a + place to perform setup that must happen in the child's context, + such as closing inherited file descriptors. - The upper bound for the fd scan comes from sysconf(_SC_OPEN_MAX), - capped at 4096 to avoid excessive iteration when the limit is set - very high. + Provide close_fd_above_stderr() as a ready-made callback that + closes file descriptors 3 and above (skipping the child-notifier + pipe), capped at sysconf(_SC_OPEN_MAX) or 4096, whichever is + smaller. Signed-off-by: Paul Tarjan <github@paulisageek.com> ## run-command.c ## +@@ run-command.c: static void trace_run_command(const struct child_process *cp) + strbuf_release(&buf); + } + ++void close_fd_above_stderr(void) ++{ ++ long max_fd = sysconf(_SC_OPEN_MAX); ++ int fd; ++ if (max_fd < 0 || max_fd > 4096) ++ max_fd = 4096; ++ for (fd = 3; fd < max_fd; fd++) { ++ if (fd != child_notifier) ++ close(fd); ++ } ++} ++ + int start_command(struct child_process *cmd) + { + int need_in, need_out, need_err; @@ run-command.c: fail_pipe: child_close(cmd->out); } -+ if (cmd->close_fd_above_stderr) { -+ long max_fd = sysconf(_SC_OPEN_MAX); -+ int fd; -+ if (max_fd < 0 || max_fd > 4096) -+ max_fd = 4096; -+ for (fd = 3; fd < max_fd; fd++) { -+ if (fd != child_notifier) -+ close(fd); -+ } -+ } ++ if (cmd->pre_exec_cb) ++ cmd->pre_exec_cb(); + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); @@ run-command.h: struct child_process { unsigned wait_after_clean:1; + + /** -+ * Close file descriptors 3 and above in the child after forking -+ * but before exec. This prevents the long-running child from -+ * inheriting pipe endpoints or other descriptors from the parent -+ * environment (e.g., the test harness). ++ * If set, the callback is invoked in the child between fork and ++ * exec. It can be used, for example, to close inherited file ++ * descriptors that the child should not keep open. + */ -+ unsigned close_fd_above_stderr:1; ++ void (*pre_exec_cb)(void); + void (*clean_on_exit_handler)(struct child_process *process); }; +@@ run-command.h: struct child_process { + .env = STRVEC_INIT, \ + } + ++/** ++ * Close file descriptors 3 and above. Suitable for use as a ++ * pre_exec_cb to prevent the child from inheriting pipe endpoints ++ * or other descriptors from the parent environment. ++ */ ++void close_fd_above_stderr(void); ++ + /** + * The functions: start_command, finish_command, run_command do the following: + * 9: b596c5004d ! 9: c963074cbd fsmonitor: close inherited file descriptors and detach in daemon @@ builtin/fsmonitor--daemon.c: static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; -+ cp.close_fd_above_stderr = 1; ++ cp.pre_exec_cb = close_fd_above_stderr; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); @@ fsmonitor-ipc.c: static int spawn_daemon(void) cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; -+ cmd.close_fd_above_stderr = 1; ++ cmd.pre_exec_cb = close_fd_above_stderr; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); 10: 72125ac20f = 10: ee3ee75c94 fsmonitor: add timeout to daemon stop command 11: 76b171ab5c = 11: 54bd8f604a fsmonitor: add tests for Linux 12: 6c36c9e11e = 12: e603fc7dde fsmonitor: convert shown khash to strset in do_handle_client -- gitgitgadget ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v9 01/12] fsmonitor: fix khash memory leak in do_handle_client 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget ` (11 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `shown` kh_str_t was freed with kh_release_str() at a point in the code only reachable in the non-trivial response path. When the client receives a trivial response, the code jumps to the `cleanup` label, skipping the kh_release_str() call entirely and leaking the hash table. Fix this by initializing `shown` to NULL and moving the cleanup to the `cleanup` label using kh_destroy_str(), which is safe to call on NULL. This ensures the hash table is freed regardless of which code path is taken. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..bc4571938c 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget ` (10 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `state.cookies` hashmap is initialized during daemon startup but never freed during cleanup in the `done:` label of fsmonitor_run_daemon(). The cookie entries also have names allocated via strbuf_detach() that must be freed individually. Iterate the hashmap to free each cookie name, then call hashmap_clear_and_free() to release the entries and table. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index bc4571938c..d8d32b01ef 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1404,6 +1404,15 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + { + struct hashmap_iter iter; + struct fsmonitor_cookie_item *cookie; + + hashmap_for_each_entry(&state.cookies, &iter, cookie, entry) + free(cookie->name); + hashmap_clear_and_free(&state.cookies, + struct fsmonitor_cookie_item, entry); + } fsm_listen__dtor(&state); fsm_health__dtor(&state); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 03/12] compat/win32: add pthread_cond_timedwait 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget ` (9 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pthread_cond_timedwait() implementation to the Windows pthread compatibility layer using SleepConditionVariableCS() with a millisecond timeout computed from the absolute deadline. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/win32/pthread.c | 26 ++++++++++++++++++++++++++ compat/win32/pthread.h | 2 ++ 2 files changed, 28 insertions(+) diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c index 7e93146963..398caa9602 100644 --- a/compat/win32/pthread.c +++ b/compat/win32/pthread.c @@ -66,3 +66,29 @@ int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) return err_win_to_posix(GetLastError()); return 0; } + +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime) +{ + struct timeval now; + long long now_ms, deadline_ms; + DWORD timeout_ms; + + gettimeofday(&now, NULL); + now_ms = (long long)now.tv_sec * 1000 + now.tv_usec / 1000; + deadline_ms = (long long)abstime->tv_sec * 1000 + + abstime->tv_nsec / 1000000; + + if (deadline_ms <= now_ms) + return ETIMEDOUT; + else + timeout_ms = (DWORD)(deadline_ms - now_ms); + + if (SleepConditionVariableCS(cond, mutex, timeout_ms) == 0) { + DWORD err = GetLastError(); + if (err == ERROR_TIMEOUT) + return ETIMEDOUT; + return err_win_to_posix(err); + } + return 0; +} diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index ccacc5a53b..d80df8d12a 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -64,6 +64,8 @@ int win32_pthread_join(pthread_t *thread, void **value_ptr); pthread_t pthread_self(void); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime); static inline void NORETURN pthread_exit(void *ret) { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (2 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget ` (8 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The cookie wait in with_lock__wait_for_cookie() uses an infinite pthread_cond_wait() loop. The existing comment notes the desire to switch to pthread_cond_timedwait(), but the routine was not available in git thread-utils. On certain container or overlay filesystems, inotify watches may succeed but events are never delivered. In this case the daemon would hang indefinitely waiting for the cookie event, which in turn causes the client to hang. Replace the infinite wait with a one-second timeout using pthread_cond_timedwait(). If the timeout fires, report an error and let the client proceed with a trivial (full-scan) response rather than blocking forever. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index d8d32b01ef..c8ec7b722e 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -197,20 +197,31 @@ static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie( unlink(cookie_pathname.buf); /* - * Technically, this is an infinite wait (well, unless another - * thread sends us an abort). I'd like to change this to - * use `pthread_cond_timedwait()` and return an error/timeout - * and let the caller do the trivial response thing, but we - * don't have that routine in our thread-utils. - * - * After extensive beta testing I'm not really worried about - * this. Also note that the above open() and unlink() calls - * will cause at least two FS events on that path, so the odds - * of getting stuck are pretty slim. + * Wait for the listener thread to observe the cookie file. + * Time out after a short interval so that the client + * does not hang forever if the filesystem does not deliver + * events (e.g., on certain container/overlay filesystems + * where inotify watches succeed but events never arrive). */ - while (cookie->result == FCIR_INIT) - pthread_cond_wait(&state->cookies_cond, - &state->main_lock); + { + struct timeval now; + struct timespec ts; + int err = 0; + + gettimeofday(&now, NULL); + ts.tv_sec = now.tv_sec + 1; + ts.tv_nsec = now.tv_usec * 1000; + + while (cookie->result == FCIR_INIT && !err) + err = pthread_cond_timedwait(&state->cookies_cond, + &state->main_lock, + &ts); + if (err == ETIMEDOUT && cookie->result == FCIR_INIT) { + trace_printf_key(&trace_fsmonitor, + "cookie_wait timed out"); + cookie->result = FCIR_ERROR; + } + } done: hashmap_remove(&state->cookies, &cookie->entry, NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (3 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget ` (7 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor IPC path logic in fsm-ipc-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-ipc-unix.c to reflect that it is shared by all Unix platforms. Introduce FSMONITOR_OS_SETTINGS (set to "unix" for non-Windows, "win32" for Windows) as a separate variable from FSMONITOR_DAEMON_BACKEND so that the build files can distinguish between platform-specific files (listen, health, path-utils) and shared Unix files (ipc, settings). Move fsm-ipc to the FSMONITOR_OS_SETTINGS section in the Makefile, and switch fsm-path-utils to use FSMONITOR_DAEMON_BACKEND since path-utils is platform-specific (there will be separate darwin and linux versions). Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 6 ++--- .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 config.mak.uname | 2 +- contrib/buildsystems/CMakeLists.txt | 25 +++++++++---------- meson.build | 7 ++++-- 5 files changed, 21 insertions(+), 19 deletions(-) rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) diff --git a/Makefile b/Makefile index 8aa489f3b6..080d009bf0 100644 --- a/Makefile +++ b/Makefile @@ -417,7 +417,7 @@ include shared.mak # If your platform has OS-specific ways to tell if a repo is incompatible with # fsmonitor (whether the hook or IPC daemon version), set FSMONITOR_OS_SETTINGS # to the "<name>" of the corresponding `compat/fsmonitor/fsm-settings-<name>.c` -# that implements the `fsm_os_settings__*()` routines. +# and `compat/fsmonitor/fsm-ipc-<name>.c` files. # # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test # programs in oss-fuzz/. @@ -2365,13 +2365,13 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o - COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef WITH_BREAKING_CHANGES diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-unix.c similarity index 100% rename from compat/fsmonitor/fsm-ipc-darwin.c rename to compat/fsmonitor/fsm-ipc-unix.c diff --git a/config.mak.uname b/config.mak.uname index 3c35ae33a3..33877020e9 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -165,7 +165,7 @@ ifeq ($(uname_S),Darwin) ifndef NO_PTHREADS ifndef NO_UNIX_SOCKETS FSMONITOR_DAEMON_BACKEND = darwin - FSMONITOR_OS_SETTINGS = darwin + FSMONITOR_OS_SETTINGS = unix endif endif diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..6197d5729c 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -291,23 +291,22 @@ endif() if(SUPPORTS_SIMPLE_IPC) if(CMAKE_SYSTEM_NAME STREQUAL "Windows") - add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) - - add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) + set(FSMONITOR_DAEMON_BACKEND "win32") + set(FSMONITOR_OS_SETTINGS "win32") elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") + set(FSMONITOR_DAEMON_BACKEND "darwin") + set(FSMONITOR_OS_SETTINGS "unix") + endif() + + if(FSMONITOR_DAEMON_BACKEND) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-${FSMONITOR_OS_SETTINGS}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) endif() endif() diff --git a/meson.build b/meson.build index dd52efd1c8..86a68365a9 100644 --- a/meson.build +++ b/meson.build @@ -1320,10 +1320,13 @@ else endif fsmonitor_backend = '' +fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' + fsmonitor_os = 'win32' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' + fsmonitor_os = 'unix' libgit_dependencies += dependency('CoreServices') endif if fsmonitor_backend != '' @@ -1332,14 +1335,14 @@ if fsmonitor_backend != '' libgit_sources += [ 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) +build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_os) if not get_option('b_sanitize').contains('address') and get_option('regex').allowed() and compiler.has_header('regex.h') and compiler.get_define('REG_STARTEND', prefix: '#include <regex.h>') != '' build_options_config.set('NO_REGEX', '') -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (4 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor settings logic in fsm-settings-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-settings-unix.c to reflect that it is shared by all Unix platforms. Update the build files (meson.build and CMakeLists.txt) to use FSMONITOR_OS_SETTINGS for fsm-settings, matching the approach already used for fsm-ipc. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} | 0 contrib/buildsystems/CMakeLists.txt | 2 +- meson.build | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-unix.c similarity index 100% rename from compat/fsmonitor/fsm-settings-darwin.c rename to compat/fsmonitor/fsm-settings-unix.c diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 6197d5729c..d613809e26 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -306,7 +306,7 @@ if(SUPPORTS_SIMPLE_IPC) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) endif() endif() diff --git a/meson.build b/meson.build index 86a68365a9..4f0c0a33b8 100644 --- a/meson.build +++ b/meson.build @@ -1338,7 +1338,7 @@ if fsmonitor_backend != '' 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 07/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (5 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 08/12] run-command: add pre-exec callback for child processes Paul Tarjan via GitGitGadget ` (5 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS's FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify's cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 ++++++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 8 +- meson.build | 4 + 8 files changed, 1042 insertions(+), 8 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..e3dca14b62 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,746 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <sys/inotify.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + if (errno == ENOSPC) + return error(_("inotify watch limit reached; " + "increase fs.inotify.max_user_watches")); + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->fd_inotify = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd >= 0 && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + /* + * Interval in seconds between checks for stale directory renames. + * A directory rename that is not completed within this window + * (i.e. no matching IN_MOVED_TO for an IN_MOVED_FROM) indicates + * missed events, forcing a shutdown. + */ + const int interval = 1; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 50); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..c9866b1b24 --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,217 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Map filesystem magic numbers to human-readable names as a fallback + * when /proc/mounts is unavailable. This only covers the remote and + * special filesystems in is_remote_fs() above; local filesystems are + * never flagged as incompatible, so we do not need their names here. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * + * statfs(2) gives us f_type (the magic number) but not the human-readable + * filesystem type string. We scan /proc/mounts to find the mount entry + * whose path is the longest prefix of ours and whose f_fsid matches, + * which gives us the fstype string (e.g. "nfs", "ext4") for logging. + */ +static char *find_mount(const char *path, const struct statfs *path_fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs->f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/config.mak.uname b/config.mak.uname index 33877020e9..638f7e1bde 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = unix + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index d613809e26..b7da108f29 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -296,6 +296,10 @@ if(SUPPORTS_SIMPLE_IPC) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") set(FSMONITOR_DAEMON_BACKEND "darwin") set(FSMONITOR_OS_SETTINGS "unix") + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + set(FSMONITOR_DAEMON_BACKEND "linux") + set(FSMONITOR_OS_SETTINGS "unix") + add_compile_definitions(HAVE_LINUX_MAGIC_H) endif() if(FSMONITOR_DAEMON_BACKEND) @@ -1149,8 +1153,8 @@ endif() file(STRINGS ${CMAKE_SOURCE_DIR}/GIT-BUILD-OPTIONS.in git_build_options NEWLINE_CONSUME) string(REPLACE "@BROKEN_PATH_FIX@" "" git_build_options "${git_build_options}") string(REPLACE "@DIFF@" "'${DIFF}'" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "win32" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_OS_SETTINGS@" "win32" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "${FSMONITOR_DAEMON_BACKEND}" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_OS_SETTINGS@" "${FSMONITOR_OS_SETTINGS}" git_build_options "${git_build_options}") string(REPLACE "@GITWEBDIR@" "'${GITWEBDIR}'" git_build_options "${git_build_options}") string(REPLACE "@GIT_INTEROP_MAKE_OPTS@" "" git_build_options "${git_build_options}") string(REPLACE "@GIT_PERF_LARGE_REPO@" "" git_build_options "${git_build_options}") diff --git a/meson.build b/meson.build index 4f0c0a33b8..123d218460 100644 --- a/meson.build +++ b/meson.build @@ -1324,6 +1324,10 @@ fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' fsmonitor_os = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + fsmonitor_os = 'unix' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' fsmonitor_os = 'unix' -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 08/12] run-command: add pre-exec callback for child processes 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget ` (4 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pre_exec_cb function pointer to struct child_process that is invoked in the child between fork and exec. This gives callers a place to perform setup that must happen in the child's context, such as closing inherited file descriptors. Provide close_fd_above_stderr() as a ready-made callback that closes file descriptors 3 and above (skipping the child-notifier pipe), capped at sysconf(_SC_OPEN_MAX) or 4096, whichever is smaller. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- run-command.c | 15 +++++++++++++++ run-command.h | 15 +++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/run-command.c b/run-command.c index e3e02475cc..b9bc84ca1b 100644 --- a/run-command.c +++ b/run-command.c @@ -674,6 +674,18 @@ static void trace_run_command(const struct child_process *cp) strbuf_release(&buf); } +void close_fd_above_stderr(void) +{ + long max_fd = sysconf(_SC_OPEN_MAX); + int fd; + if (max_fd < 0 || max_fd > 4096) + max_fd = 4096; + for (fd = 3; fd < max_fd; fd++) { + if (fd != child_notifier) + close(fd); + } +} + int start_command(struct child_process *cmd) { int need_in, need_out, need_err; @@ -832,6 +844,9 @@ fail_pipe: child_close(cmd->out); } + if (cmd->pre_exec_cb) + cmd->pre_exec_cb(); + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); diff --git a/run-command.h b/run-command.h index 0df25e445f..7ea5c6e005 100644 --- a/run-command.h +++ b/run-command.h @@ -141,6 +141,14 @@ struct child_process { unsigned stdout_to_stderr:1; unsigned clean_on_exit:1; unsigned wait_after_clean:1; + + /** + * If set, the callback is invoked in the child between fork and + * exec. It can be used, for example, to close inherited file + * descriptors that the child should not keep open. + */ + void (*pre_exec_cb)(void); + void (*clean_on_exit_handler)(struct child_process *process); }; @@ -149,6 +157,13 @@ struct child_process { .env = STRVEC_INIT, \ } +/** + * Close file descriptors 3 and above. Suitable for use as a + * pre_exec_cb to prevent the child from inheriting pipe endpoints + * or other descriptors from the parent environment. + */ +void close_fd_above_stderr(void); + /** * The functions: start_command, finish_command, run_command do the following: * -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 09/12] fsmonitor: close inherited file descriptors and detach in daemon 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (7 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 08/12] run-command: add pre-exec callback for child processes Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget ` (3 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> When the fsmonitor daemon is spawned as a background process, it may inherit file descriptors from its parent that it does not need. In particular, when the test harness or a CI system captures output through pipes, the daemon can inherit duplicated pipe endpoints. If the daemon holds these open, the parent process never sees EOF and may appear to hang. Set close_fd_above_stderr on the child process at both daemon startup paths: the explicit "fsmonitor--daemon start" command and the implicit spawn triggered by fsmonitor-ipc when a client finds no running daemon. Also suppress stdout and stderr on the implicit spawn path to prevent the background daemon from writing to the client's terminal. Additionally, call setsid() when the daemon starts with --detach to create a new session and process group. This prevents the daemon from being part of the spawning shell's process group, which could cause the shell's "wait" to block until the daemon exits. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 16 ++++++++++++++-- fsmonitor-ipc.c | 3 +++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index c8ec7b722e..d2f250bd06 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1439,7 +1439,7 @@ done: return err; } -static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) +static int try_to_run_foreground_daemon(int detach_console) { /* * Technically, we don't need to probe for an existing daemon @@ -1459,10 +1459,21 @@ static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) fflush(stderr); } + if (detach_console) { #ifdef GIT_WINDOWS_NATIVE - if (detach_console) FreeConsole(); +#else + /* + * Create a new session so that the daemon is detached + * from the parent's process group. This prevents + * shells with job control (e.g. bash with "set -m") + * from waiting on the daemon when they wait for a + * foreground command that implicitly spawned it. + */ + if (setsid() == -1) + warning_errno(_("setsid failed")); #endif + } return !!fsmonitor_run_daemon(); } @@ -1525,6 +1536,7 @@ static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; + cp.pre_exec_cb = close_fd_above_stderr; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c index f1b1631111..61cb789339 100644 --- a/fsmonitor-ipc.c +++ b/fsmonitor-ipc.c @@ -61,6 +61,9 @@ static int spawn_daemon(void) cmd.git_cmd = 1; cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; + cmd.pre_exec_cb = close_fd_above_stderr; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 10/12] fsmonitor: add timeout to daemon stop command 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (8 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget ` (2 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The "fsmonitor--daemon stop" command polls in a loop waiting for the daemon to exit after sending a "quit" command over IPC. If the daemon fails to shut down (e.g. it is stuck or wedged), this loop spins forever. Add a 30-second timeout so the stop command returns an error instead of blocking indefinitely. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index d2f250bd06..299de2e4e2 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -86,6 +86,8 @@ static int do_as_client__send_stop(void) { struct strbuf answer = STRBUF_INIT; int ret; + int max_wait_ms = 30000; + int elapsed_ms = 0; ret = fsmonitor_ipc__send_command("quit", &answer); @@ -96,8 +98,16 @@ static int do_as_client__send_stop(void) return ret; trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL); - while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) + while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) { + if (elapsed_ms >= max_wait_ms) { + trace2_region_leave("fsm_client", + "polling-for-daemon-exit", NULL); + return error(_("daemon did not stop within %d seconds"), + max_wait_ms / 1000); + } sleep_millisec(50); + elapsed_ms += 50; + } trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL); return 0; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 11/12] fsmonitor: add tests for Linux 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (9 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 ` Paul Tarjan via GitGitGadget 2026-03-05 0:52 ` [PATCH v9 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:51 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a smoke test that verifies the filesystem actually delivers inotify events to the daemon. On some configurations (e.g., overlayfs with older kernels), inotify watches succeed but events are never delivered. The daemon cookie wait will time out, but every subsequent test would fail. Skip the entire test file early when this is detected. Add a test that exercises rapid nested directory creation to verify the daemon correctly handles the EEXIST race between recursive scan and queued inotify events. When IN_MASK_CREATE is available and a directory watch is added during recursive registration, the kernel may also deliver a queued IN_CREATE event for the same directory. The second inotify_add_watch() returns EEXIST, which must be treated as harmless. An earlier version of the listener crashed in this scenario. Reduce --start-timeout from the default 60 seconds to 10 seconds so that tests fail promptly when the daemon cannot start. Harden the test helpers to work in environments without procps (e.g., Fedora CI): fall back to reading /proc/$pid/stat for the process group ID when ps is unavailable, guard stop_git() against an empty pgid, and redirect stderr from kill to /dev/null to avoid noise when processes have already exited. Use set -m to enable job control in the submodule-pull test so that the background git pull gets its own process group, preventing the shell wait from blocking on the daemon. setsid() in the previous commit detaches the daemon itself, but the intermediate git pull process still needs its own process group for the test shell to manage it correctly. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- t/t7527-builtin-fsmonitor.sh | 89 +++++++++++++++++++++++++++++++++--- 1 file changed, 82 insertions(+), 7 deletions(-) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..774da5ac60 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -10,9 +10,58 @@ then test_done fi +# Verify that the filesystem delivers events to the daemon. +# On some configurations (e.g., overlayfs with older kernels), +# inotify watches succeed but events are never delivered. The +# cookie wait will time out and the daemon logs a trace message. +# +# Use "timeout" (if available) to guard each step against hangs. +maybe_timeout () { + if type timeout >/dev/null 2>&1 + then + timeout "$@" + else + shift + "$@" + fi +} +verify_fsmonitor_works () { + git init test_fsmonitor_smoke || return 1 + + GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && + export GIT_TRACE_FSMONITOR && + maybe_timeout 30 \ + git -C test_fsmonitor_smoke fsmonitor--daemon start \ + --start-timeout=10 + ret=$? + unset GIT_TRACE_FSMONITOR + if test $ret -ne 0 + then + rm -rf test_fsmonitor_smoke smoke.trace + return 1 + fi + + maybe_timeout 10 \ + test-tool -C test_fsmonitor_smoke fsmonitor-client query \ + --token 0 >/dev/null 2>&1 + maybe_timeout 5 \ + git -C test_fsmonitor_smoke fsmonitor--daemon stop 2>/dev/null + ! grep -q "cookie_wait timed out" "$PWD/smoke.trace" 2>/dev/null + ret=$? + rm -rf test_fsmonitor_smoke smoke.trace + return $ret +} + +if ! verify_fsmonitor_works +then + skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" + test_done +fi + stop_daemon_delete_repo () { r=$1 && - test_might_fail git -C $r fsmonitor--daemon stop && + test_might_fail maybe_timeout 30 \ + git -C $r fsmonitor--daemon stop 2>/dev/null rm -rf $1 } @@ -67,7 +116,7 @@ start_daemon () { export GIT_TEST_FSMONITOR_TOKEN fi && - git $r fsmonitor--daemon start && + git $r fsmonitor--daemon start --start-timeout=10 && git $r fsmonitor--daemon status ) } @@ -520,6 +569,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the @@ -910,7 +981,10 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" ' start_git_in_background () { git "$@" & git_pid=$! - git_pgid=$(ps -o pgid= -p $git_pid) + git_pgid=$(ps -o pgid= -p $git_pid 2>/dev/null || + awk '{print $5}' /proc/$git_pid/stat 2>/dev/null) && + git_pgid="${git_pgid## }" && + git_pgid="${git_pgid%% }" nr_tries_left=10 while true do @@ -921,15 +995,16 @@ start_git_in_background () { fi sleep 1 nr_tries_left=$(($nr_tries_left - 1)) - done >/dev/null 2>&1 & + done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! wait $git_pid } stop_git () { - while kill -0 -- -$git_pgid + test -n "$git_pgid" || return 0 + while kill -0 -- -$git_pgid 2>/dev/null do - kill -- -$git_pgid + kill -- -$git_pgid 2>/dev/null sleep 1 done } @@ -944,7 +1019,7 @@ stop_watchdog () { test_expect_success !MINGW "submodule implicitly starts daemon by pull" ' test_atexit "stop_watchdog" && - test_when_finished "stop_git; rm -rf cloned super sub" && + test_when_finished "set +m; stop_git; rm -rf cloned super sub" && create_super super && create_sub sub && -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v9 12/12] fsmonitor: convert shown khash to strset in do_handle_client 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (10 preceding siblings ...) 2026-03-05 0:51 ` [PATCH v9 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 0:52 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 0:52 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Replace the khash-based string set used for deduplicating pathnames in do_handle_client() with a strset, which provides a cleaner interface for the same purpose. Since the paths are interned strings from the batch data, use strdup_strings=0 to avoid unnecessary copies. Suggested-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 299de2e4e2..b4b2a304e5 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -16,7 +16,7 @@ #include "fsmonitor--daemon.h" #include "simple-ipc.h" -#include "khash.h" +#include "strmap.h" #include "run-command.h" #include "trace.h" #include "trace2.h" @@ -674,8 +674,6 @@ static int fsmonitor_parse_client_token(const char *buf_token, return 0; } -KHASH_INIT(str, const char *, int, 0, kh_str_hash_func, kh_str_hash_equal) - static int do_handle_client(struct fsmonitor_daemon_state *state, const char *command, ipc_server_reply_cb *reply, @@ -692,8 +690,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown = NULL; - int hash_ret; + struct strset shown = STRSET_INIT; int do_trivial = 0; int do_flush = 0; int do_cookie = 0; @@ -882,14 +879,14 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, * so walk the batch list backwards from the current head back * to the batch (sequence number) they named. * - * We use khash to de-dup the list of pathnames. + * We use a strset to de-dup the list of pathnames. * * NEEDSWORK: each batch contains a list of interned strings, * so we only need to do pointer comparisons here to build the * hash table. Currently, we're still comparing the string * values. */ - shown = kh_init_str(); + strset_init_with_options(&shown, NULL, 0); for (batch = batch_head; batch && batch->batch_seq_nr > requested_oldest_seq_nr; batch = batch->next) { @@ -899,11 +896,9 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const char *s = batch->interned_paths[k]; size_t s_len; - if (kh_get_str(shown, s) != kh_end(shown)) + if (!strset_add(&shown, s)) duplicates++; else { - kh_put_str(shown, s, &hash_ret); - trace_printf_key(&trace_fsmonitor, "send[%"PRIuMAX"]: %s", count, s); @@ -973,7 +968,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: - kh_destroy_str(shown); + strset_clear(&shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (11 preceding siblings ...) 2026-03-05 0:52 ` [PATCH v9 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget ` (12 more replies) 12 siblings, 13 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan This series implements the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. It also fixes two memory leaks in the platform-independent daemon code and deduplicates the IPC and settings logic that is now shared between macOS and Linux. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. This series builds on work from https://github.com/git/git/pull/1352 by Eric DeCosta and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v8: * Replaced close_fd_above_stderr flag with generic pre_exec_cb callback in struct child_process per Junio's review * Provided close_fd_above_stderr() as a ready-made callback function * Fixed Windows build: close_fd_above_stderr() compiles as a no-op on Windows since there is no fork/exec Changes since v7: * Added patch 12: convert khash to strset in do_handle_client (Patrick's #leftoverbit suggestion) * Fixed "Forcing shutdown" trace message to start with lowercase * Fixed redundant statfs() call in find_mount() (caller already had the result) * Fixed CMakeLists.txt GIT-BUILD-OPTIONS: was hardcoded to "win32" for FSMONITOR_DAEMON_BACKEND and FSMONITOR_OS_SETTINGS, now uses the CMake variables * Fixed uninitialized strset on trivial response path (STRSET_INIT) * Removed V9FS_MAGIC from get_fs_typename() to match is_remote_fs() (9p is local VM mounts) * Split 30-second stop timeout into its own commit per review request * Fixed misleading indentation on shutdown assignment in handle_events() * Updated commit messages to describe all changes (test hardening, fsmonitor-ipc.c spawn changes) * Updated Makefile comment for FSMONITOR_OS_SETTINGS to mention fsm-ipc Changes since v6: * Introduced FSMONITOR_OS_SETTINGS build variable (set to "unix" for macOS and Linux, "win32" for Windows) to eliminate if/else conditionals in Makefile, meson.build, and CMakeLists.txt per Junio's review * Moved fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND since path-utils files are platform-specific * Removed V9FS_MAGIC from remote filesystem detection (9p is used for local VM/container host mounts where fsmonitor works fine) * Removed redundant #include <libgen.h> (already provided by compat/posix.h) * Fixed cookie wait comment wording ("see" to "observe") * Rewrote commit messages for IPC and settings dedup patches Changes since v5: * Split monolithic commit into 10-patch series per Patrick's review * Deduplicated fsm-ipc and fsm-settings into shared Unix implementations * Rewrote commit message with prose paragraphs, explain inotify vs fanotify, removed "Issues addressed" sections, added Based-on-patch-by trailers * Removed redundant includes already provided by compat/posix.h * Fixed error/trace message capitalization per coding guidelines * Fixed stale rename check interval from 1000 seconds to 1 second * Changed poll timeout from 1ms to 50ms to reduce idle CPU wake-ups * Replaced infinite pthread_cond_wait cookie loop with one-second pthread_cond_timedwait (prevents daemon hangs on overlay filesystems where events are never delivered) * Added pthread_cond_timedwait to Windows pthread compatibility layer * Separated test into its own commit with smoke test that skips when inotify events are not delivered (e.g., overlayfs with older kernels) * Fixed test hang on Fedora CI: stop_git() looped forever when ps was unavailable because bash in POSIX/sh mode returns exit 0 from kill with an empty process group argument. Fixed by falling back to /proc/$pid/stat for process group ID and guarding stop_git against empty pgid. * Redirect spawn_daemon() stdout/stderr to /dev/null and close inherited file descriptors to prevent the intermediate process from holding test pipe file descriptors * Call setsid() on daemon detach to prevent shells with job control from waiting on the daemon process group * Close inherited file descriptors 3-7 in the test watchdog subprocess * Added 30-second timeout to "fsmonitor--daemon stop" to prevent indefinite blocking * Added helpful error message when inotify watch limit (max_user_watches) is reached * Initialize fd_inotify to -1 and use fd >= 0 check for correct fd 0 handling * Use sysconf(_SC_OPEN_MAX) instead of hardcoded 1024 for fd close limit * Check setsid() return value Changes since v4: * Added Meson build support Changes since v3: * Fix crash on rapid nested directory creation (EEXIST from inotify_add_watch with IN_MASK_CREATE) * Extensive stress testing Changes since v2: * Fix khash memory leak in do_handle_client Changes since v1: * Fix hashmap memory leak in fsmonitor_run_daemon() Paul Tarjan (12): fsmonitor: fix khash memory leak in do_handle_client fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon compat/win32: add pthread_cond_timedwait fsmonitor: use pthread_cond_timedwait for cookie wait fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c fsmonitor: implement filesystem change listener for Linux run-command: add pre-exec callback for child processes fsmonitor: close inherited file descriptors and detach in daemon fsmonitor: add timeout to daemon stop command fsmonitor: add tests for Linux fsmonitor: convert shown khash to strset in do_handle_client Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- Makefile | 6 +- builtin/fsmonitor--daemon.c | 92 ++- compat/fsmonitor/fsm-health-linux.c | 33 + .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 +++++ ...-settings-darwin.c => fsm-settings-unix.c} | 0 compat/win32/pthread.c | 26 + compat/win32/pthread.h | 2 + config.mak.uname | 12 +- contrib/buildsystems/CMakeLists.txt | 33 +- fsmonitor-ipc.c | 3 + meson.build | 13 +- run-command.c | 18 + run-command.h | 16 + t/t7527-builtin-fsmonitor.sh | 89 ++- 18 files changed, 1275 insertions(+), 63 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) base-commit: 3e0db84c88c57e70ac8be8c196dfa92c5d656fbc Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v10 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v10 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v9: 1: 4d4dec8fa1 = 1: 4d4dec8fa1 fsmonitor: fix khash memory leak in do_handle_client 2: cb270120f0 = 2: cb270120f0 fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 3: 44a063074d = 3: 44a063074d compat/win32: add pthread_cond_timedwait 4: b1081d1e13 = 4: b1081d1e13 fsmonitor: use pthread_cond_timedwait for cookie wait 5: dec0fb144f = 5: dec0fb144f fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c 6: b2aaadb4ae = 6: b2aaadb4ae fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c 7: 03cf12d01b = 7: 03cf12d01b fsmonitor: implement filesystem change listener for Linux 8: 31fa1fb324 ! 8: 39dcfbb7c8 run-command: add pre-exec callback for child processes @@ Commit message Signed-off-by: Paul Tarjan <github@paulisageek.com> ## run-command.c ## -@@ run-command.c: static void trace_run_command(const struct child_process *cp) - strbuf_release(&buf); +@@ run-command.c: static void atfork_parent(struct atfork_state *as) + "restoring signal mask"); + #endif } ++ + #endif /* GIT_WINDOWS_NATIVE */ +void close_fd_above_stderr(void) +{ ++#ifndef GIT_WINDOWS_NATIVE + long max_fd = sysconf(_SC_OPEN_MAX); + int fd; + if (max_fd < 0 || max_fd > 4096) @@ run-command.c: static void trace_run_command(const struct child_process *cp) + if (fd != child_notifier) + close(fd); + } ++#endif +} + - int start_command(struct child_process *cmd) + static inline void set_cloexec(int fd) { - int need_in, need_out, need_err; + int flags = fcntl(fd, F_GETFD); @@ run-command.c: fail_pipe: child_close(cmd->out); } @@ run-command.h: struct child_process { +/** + * Close file descriptors 3 and above. Suitable for use as a + * pre_exec_cb to prevent the child from inheriting pipe endpoints -+ * or other descriptors from the parent environment. ++ * or other descriptors from the parent environment. On Windows ++ * this is a no-op since there is no fork/exec. + */ +void close_fd_above_stderr(void); + 9: c963074cbd = 9: 5db0591c15 fsmonitor: close inherited file descriptors and detach in daemon 10: ee3ee75c94 = 10: 8a9a6ba4fa fsmonitor: add timeout to daemon stop command 11: 54bd8f604a = 11: 27d5560007 fsmonitor: add tests for Linux 12: e603fc7dde = 12: 8ea20aab4c fsmonitor: convert shown khash to strset in do_handle_client -- gitgitgadget ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v10 01/12] fsmonitor: fix khash memory leak in do_handle_client 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget ` (11 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `shown` kh_str_t was freed with kh_release_str() at a point in the code only reachable in the non-trivial response path. When the client receives a trivial response, the code jumps to the `cleanup` label, skipping the kh_release_str() call entirely and leaking the hash table. Fix this by initializing `shown` to NULL and moving the cleanup to the `cleanup` label using kh_destroy_str(), which is safe to call on NULL. This ensures the hash table is freed regardless of which code path is taken. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..bc4571938c 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget ` (10 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `state.cookies` hashmap is initialized during daemon startup but never freed during cleanup in the `done:` label of fsmonitor_run_daemon(). The cookie entries also have names allocated via strbuf_detach() that must be freed individually. Iterate the hashmap to free each cookie name, then call hashmap_clear_and_free() to release the entries and table. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index bc4571938c..d8d32b01ef 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1404,6 +1404,15 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + { + struct hashmap_iter iter; + struct fsmonitor_cookie_item *cookie; + + hashmap_for_each_entry(&state.cookies, &iter, cookie, entry) + free(cookie->name); + hashmap_clear_and_free(&state.cookies, + struct fsmonitor_cookie_item, entry); + } fsm_listen__dtor(&state); fsm_health__dtor(&state); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 03/12] compat/win32: add pthread_cond_timedwait 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget ` (9 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pthread_cond_timedwait() implementation to the Windows pthread compatibility layer using SleepConditionVariableCS() with a millisecond timeout computed from the absolute deadline. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/win32/pthread.c | 26 ++++++++++++++++++++++++++ compat/win32/pthread.h | 2 ++ 2 files changed, 28 insertions(+) diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c index 7e93146963..398caa9602 100644 --- a/compat/win32/pthread.c +++ b/compat/win32/pthread.c @@ -66,3 +66,29 @@ int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) return err_win_to_posix(GetLastError()); return 0; } + +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime) +{ + struct timeval now; + long long now_ms, deadline_ms; + DWORD timeout_ms; + + gettimeofday(&now, NULL); + now_ms = (long long)now.tv_sec * 1000 + now.tv_usec / 1000; + deadline_ms = (long long)abstime->tv_sec * 1000 + + abstime->tv_nsec / 1000000; + + if (deadline_ms <= now_ms) + return ETIMEDOUT; + else + timeout_ms = (DWORD)(deadline_ms - now_ms); + + if (SleepConditionVariableCS(cond, mutex, timeout_ms) == 0) { + DWORD err = GetLastError(); + if (err == ERROR_TIMEOUT) + return ETIMEDOUT; + return err_win_to_posix(err); + } + return 0; +} diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index ccacc5a53b..d80df8d12a 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -64,6 +64,8 @@ int win32_pthread_join(pthread_t *thread, void **value_ptr); pthread_t pthread_self(void); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime); static inline void NORETURN pthread_exit(void *ret) { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (2 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget ` (8 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The cookie wait in with_lock__wait_for_cookie() uses an infinite pthread_cond_wait() loop. The existing comment notes the desire to switch to pthread_cond_timedwait(), but the routine was not available in git thread-utils. On certain container or overlay filesystems, inotify watches may succeed but events are never delivered. In this case the daemon would hang indefinitely waiting for the cookie event, which in turn causes the client to hang. Replace the infinite wait with a one-second timeout using pthread_cond_timedwait(). If the timeout fires, report an error and let the client proceed with a trivial (full-scan) response rather than blocking forever. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index d8d32b01ef..c8ec7b722e 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -197,20 +197,31 @@ static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie( unlink(cookie_pathname.buf); /* - * Technically, this is an infinite wait (well, unless another - * thread sends us an abort). I'd like to change this to - * use `pthread_cond_timedwait()` and return an error/timeout - * and let the caller do the trivial response thing, but we - * don't have that routine in our thread-utils. - * - * After extensive beta testing I'm not really worried about - * this. Also note that the above open() and unlink() calls - * will cause at least two FS events on that path, so the odds - * of getting stuck are pretty slim. + * Wait for the listener thread to observe the cookie file. + * Time out after a short interval so that the client + * does not hang forever if the filesystem does not deliver + * events (e.g., on certain container/overlay filesystems + * where inotify watches succeed but events never arrive). */ - while (cookie->result == FCIR_INIT) - pthread_cond_wait(&state->cookies_cond, - &state->main_lock); + { + struct timeval now; + struct timespec ts; + int err = 0; + + gettimeofday(&now, NULL); + ts.tv_sec = now.tv_sec + 1; + ts.tv_nsec = now.tv_usec * 1000; + + while (cookie->result == FCIR_INIT && !err) + err = pthread_cond_timedwait(&state->cookies_cond, + &state->main_lock, + &ts); + if (err == ETIMEDOUT && cookie->result == FCIR_INIT) { + trace_printf_key(&trace_fsmonitor, + "cookie_wait timed out"); + cookie->result = FCIR_ERROR; + } + } done: hashmap_remove(&state->cookies, &cookie->entry, NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (3 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget ` (7 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor IPC path logic in fsm-ipc-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-ipc-unix.c to reflect that it is shared by all Unix platforms. Introduce FSMONITOR_OS_SETTINGS (set to "unix" for non-Windows, "win32" for Windows) as a separate variable from FSMONITOR_DAEMON_BACKEND so that the build files can distinguish between platform-specific files (listen, health, path-utils) and shared Unix files (ipc, settings). Move fsm-ipc to the FSMONITOR_OS_SETTINGS section in the Makefile, and switch fsm-path-utils to use FSMONITOR_DAEMON_BACKEND since path-utils is platform-specific (there will be separate darwin and linux versions). Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 6 ++--- .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 config.mak.uname | 2 +- contrib/buildsystems/CMakeLists.txt | 25 +++++++++---------- meson.build | 7 ++++-- 5 files changed, 21 insertions(+), 19 deletions(-) rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) diff --git a/Makefile b/Makefile index 8aa489f3b6..080d009bf0 100644 --- a/Makefile +++ b/Makefile @@ -417,7 +417,7 @@ include shared.mak # If your platform has OS-specific ways to tell if a repo is incompatible with # fsmonitor (whether the hook or IPC daemon version), set FSMONITOR_OS_SETTINGS # to the "<name>" of the corresponding `compat/fsmonitor/fsm-settings-<name>.c` -# that implements the `fsm_os_settings__*()` routines. +# and `compat/fsmonitor/fsm-ipc-<name>.c` files. # # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test # programs in oss-fuzz/. @@ -2365,13 +2365,13 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o - COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef WITH_BREAKING_CHANGES diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-unix.c similarity index 100% rename from compat/fsmonitor/fsm-ipc-darwin.c rename to compat/fsmonitor/fsm-ipc-unix.c diff --git a/config.mak.uname b/config.mak.uname index 3c35ae33a3..33877020e9 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -165,7 +165,7 @@ ifeq ($(uname_S),Darwin) ifndef NO_PTHREADS ifndef NO_UNIX_SOCKETS FSMONITOR_DAEMON_BACKEND = darwin - FSMONITOR_OS_SETTINGS = darwin + FSMONITOR_OS_SETTINGS = unix endif endif diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..6197d5729c 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -291,23 +291,22 @@ endif() if(SUPPORTS_SIMPLE_IPC) if(CMAKE_SYSTEM_NAME STREQUAL "Windows") - add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) - - add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) + set(FSMONITOR_DAEMON_BACKEND "win32") + set(FSMONITOR_OS_SETTINGS "win32") elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") + set(FSMONITOR_DAEMON_BACKEND "darwin") + set(FSMONITOR_OS_SETTINGS "unix") + endif() + + if(FSMONITOR_DAEMON_BACKEND) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-${FSMONITOR_OS_SETTINGS}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) endif() endif() diff --git a/meson.build b/meson.build index dd52efd1c8..86a68365a9 100644 --- a/meson.build +++ b/meson.build @@ -1320,10 +1320,13 @@ else endif fsmonitor_backend = '' +fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' + fsmonitor_os = 'win32' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' + fsmonitor_os = 'unix' libgit_dependencies += dependency('CoreServices') endif if fsmonitor_backend != '' @@ -1332,14 +1335,14 @@ if fsmonitor_backend != '' libgit_sources += [ 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) +build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_os) if not get_option('b_sanitize').contains('address') and get_option('regex').allowed() and compiler.has_header('regex.h') and compiler.get_define('REG_STARTEND', prefix: '#include <regex.h>') != '' build_options_config.set('NO_REGEX', '') -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (4 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor settings logic in fsm-settings-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-settings-unix.c to reflect that it is shared by all Unix platforms. Update the build files (meson.build and CMakeLists.txt) to use FSMONITOR_OS_SETTINGS for fsm-settings, matching the approach already used for fsm-ipc. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} | 0 contrib/buildsystems/CMakeLists.txt | 2 +- meson.build | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-unix.c similarity index 100% rename from compat/fsmonitor/fsm-settings-darwin.c rename to compat/fsmonitor/fsm-settings-unix.c diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 6197d5729c..d613809e26 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -306,7 +306,7 @@ if(SUPPORTS_SIMPLE_IPC) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) endif() endif() diff --git a/meson.build b/meson.build index 86a68365a9..4f0c0a33b8 100644 --- a/meson.build +++ b/meson.build @@ -1338,7 +1338,7 @@ if fsmonitor_backend != '' 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 07/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (5 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 08/12] run-command: add pre-exec callback for child processes Paul Tarjan via GitGitGadget ` (5 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS's FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify's cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 ++++++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 8 +- meson.build | 4 + 8 files changed, 1042 insertions(+), 8 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..e3dca14b62 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,746 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <sys/inotify.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + if (errno == ENOSPC) + return error(_("inotify watch limit reached; " + "increase fs.inotify.max_user_watches")); + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->fd_inotify = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd >= 0 && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + /* + * Interval in seconds between checks for stale directory renames. + * A directory rename that is not completed within this window + * (i.e. no matching IN_MOVED_TO for an IN_MOVED_FROM) indicates + * missed events, forcing a shutdown. + */ + const int interval = 1; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 50); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..c9866b1b24 --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,217 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Map filesystem magic numbers to human-readable names as a fallback + * when /proc/mounts is unavailable. This only covers the remote and + * special filesystems in is_remote_fs() above; local filesystems are + * never flagged as incompatible, so we do not need their names here. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * + * statfs(2) gives us f_type (the magic number) but not the human-readable + * filesystem type string. We scan /proc/mounts to find the mount entry + * whose path is the longest prefix of ours and whose f_fsid matches, + * which gives us the fstype string (e.g. "nfs", "ext4") for logging. + */ +static char *find_mount(const char *path, const struct statfs *path_fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs->f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/config.mak.uname b/config.mak.uname index 33877020e9..638f7e1bde 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = unix + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index d613809e26..b7da108f29 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -296,6 +296,10 @@ if(SUPPORTS_SIMPLE_IPC) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") set(FSMONITOR_DAEMON_BACKEND "darwin") set(FSMONITOR_OS_SETTINGS "unix") + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + set(FSMONITOR_DAEMON_BACKEND "linux") + set(FSMONITOR_OS_SETTINGS "unix") + add_compile_definitions(HAVE_LINUX_MAGIC_H) endif() if(FSMONITOR_DAEMON_BACKEND) @@ -1149,8 +1153,8 @@ endif() file(STRINGS ${CMAKE_SOURCE_DIR}/GIT-BUILD-OPTIONS.in git_build_options NEWLINE_CONSUME) string(REPLACE "@BROKEN_PATH_FIX@" "" git_build_options "${git_build_options}") string(REPLACE "@DIFF@" "'${DIFF}'" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "win32" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_OS_SETTINGS@" "win32" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "${FSMONITOR_DAEMON_BACKEND}" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_OS_SETTINGS@" "${FSMONITOR_OS_SETTINGS}" git_build_options "${git_build_options}") string(REPLACE "@GITWEBDIR@" "'${GITWEBDIR}'" git_build_options "${git_build_options}") string(REPLACE "@GIT_INTEROP_MAKE_OPTS@" "" git_build_options "${git_build_options}") string(REPLACE "@GIT_PERF_LARGE_REPO@" "" git_build_options "${git_build_options}") diff --git a/meson.build b/meson.build index 4f0c0a33b8..123d218460 100644 --- a/meson.build +++ b/meson.build @@ -1324,6 +1324,10 @@ fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' fsmonitor_os = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + fsmonitor_os = 'unix' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' fsmonitor_os = 'unix' -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 08/12] run-command: add pre-exec callback for child processes 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget ` (4 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pre_exec_cb function pointer to struct child_process that is invoked in the child between fork and exec. This gives callers a place to perform setup that must happen in the child's context, such as closing inherited file descriptors. Provide close_fd_above_stderr() as a ready-made callback that closes file descriptors 3 and above (skipping the child-notifier pipe), capped at sysconf(_SC_OPEN_MAX) or 4096, whichever is smaller. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- run-command.c | 18 ++++++++++++++++++ run-command.h | 16 ++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/run-command.c b/run-command.c index e3e02475cc..cfbfe68b33 100644 --- a/run-command.c +++ b/run-command.c @@ -546,8 +546,23 @@ static void atfork_parent(struct atfork_state *as) "restoring signal mask"); #endif } + #endif /* GIT_WINDOWS_NATIVE */ +void close_fd_above_stderr(void) +{ +#ifndef GIT_WINDOWS_NATIVE + long max_fd = sysconf(_SC_OPEN_MAX); + int fd; + if (max_fd < 0 || max_fd > 4096) + max_fd = 4096; + for (fd = 3; fd < max_fd; fd++) { + if (fd != child_notifier) + close(fd); + } +#endif +} + static inline void set_cloexec(int fd) { int flags = fcntl(fd, F_GETFD); @@ -832,6 +847,9 @@ fail_pipe: child_close(cmd->out); } + if (cmd->pre_exec_cb) + cmd->pre_exec_cb(); + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); diff --git a/run-command.h b/run-command.h index 0df25e445f..375d2c731d 100644 --- a/run-command.h +++ b/run-command.h @@ -141,6 +141,14 @@ struct child_process { unsigned stdout_to_stderr:1; unsigned clean_on_exit:1; unsigned wait_after_clean:1; + + /** + * If set, the callback is invoked in the child between fork and + * exec. It can be used, for example, to close inherited file + * descriptors that the child should not keep open. + */ + void (*pre_exec_cb)(void); + void (*clean_on_exit_handler)(struct child_process *process); }; @@ -149,6 +157,14 @@ struct child_process { .env = STRVEC_INIT, \ } +/** + * Close file descriptors 3 and above. Suitable for use as a + * pre_exec_cb to prevent the child from inheriting pipe endpoints + * or other descriptors from the parent environment. On Windows + * this is a no-op since there is no fork/exec. + */ +void close_fd_above_stderr(void); + /** * The functions: start_command, finish_command, run_command do the following: * -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 09/12] fsmonitor: close inherited file descriptors and detach in daemon 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (7 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 08/12] run-command: add pre-exec callback for child processes Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget ` (3 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> When the fsmonitor daemon is spawned as a background process, it may inherit file descriptors from its parent that it does not need. In particular, when the test harness or a CI system captures output through pipes, the daemon can inherit duplicated pipe endpoints. If the daemon holds these open, the parent process never sees EOF and may appear to hang. Set close_fd_above_stderr on the child process at both daemon startup paths: the explicit "fsmonitor--daemon start" command and the implicit spawn triggered by fsmonitor-ipc when a client finds no running daemon. Also suppress stdout and stderr on the implicit spawn path to prevent the background daemon from writing to the client's terminal. Additionally, call setsid() when the daemon starts with --detach to create a new session and process group. This prevents the daemon from being part of the spawning shell's process group, which could cause the shell's "wait" to block until the daemon exits. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 16 ++++++++++++++-- fsmonitor-ipc.c | 3 +++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index c8ec7b722e..d2f250bd06 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1439,7 +1439,7 @@ done: return err; } -static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) +static int try_to_run_foreground_daemon(int detach_console) { /* * Technically, we don't need to probe for an existing daemon @@ -1459,10 +1459,21 @@ static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) fflush(stderr); } + if (detach_console) { #ifdef GIT_WINDOWS_NATIVE - if (detach_console) FreeConsole(); +#else + /* + * Create a new session so that the daemon is detached + * from the parent's process group. This prevents + * shells with job control (e.g. bash with "set -m") + * from waiting on the daemon when they wait for a + * foreground command that implicitly spawned it. + */ + if (setsid() == -1) + warning_errno(_("setsid failed")); #endif + } return !!fsmonitor_run_daemon(); } @@ -1525,6 +1536,7 @@ static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; + cp.pre_exec_cb = close_fd_above_stderr; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c index f1b1631111..61cb789339 100644 --- a/fsmonitor-ipc.c +++ b/fsmonitor-ipc.c @@ -61,6 +61,9 @@ static int spawn_daemon(void) cmd.git_cmd = 1; cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; + cmd.pre_exec_cb = close_fd_above_stderr; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 10/12] fsmonitor: add timeout to daemon stop command 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (8 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget ` (2 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The "fsmonitor--daemon stop" command polls in a loop waiting for the daemon to exit after sending a "quit" command over IPC. If the daemon fails to shut down (e.g. it is stuck or wedged), this loop spins forever. Add a 30-second timeout so the stop command returns an error instead of blocking indefinitely. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index d2f250bd06..299de2e4e2 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -86,6 +86,8 @@ static int do_as_client__send_stop(void) { struct strbuf answer = STRBUF_INIT; int ret; + int max_wait_ms = 30000; + int elapsed_ms = 0; ret = fsmonitor_ipc__send_command("quit", &answer); @@ -96,8 +98,16 @@ static int do_as_client__send_stop(void) return ret; trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL); - while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) + while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) { + if (elapsed_ms >= max_wait_ms) { + trace2_region_leave("fsm_client", + "polling-for-daemon-exit", NULL); + return error(_("daemon did not stop within %d seconds"), + max_wait_ms / 1000); + } sleep_millisec(50); + elapsed_ms += 50; + } trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL); return 0; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 11/12] fsmonitor: add tests for Linux 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (9 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a smoke test that verifies the filesystem actually delivers inotify events to the daemon. On some configurations (e.g., overlayfs with older kernels), inotify watches succeed but events are never delivered. The daemon cookie wait will time out, but every subsequent test would fail. Skip the entire test file early when this is detected. Add a test that exercises rapid nested directory creation to verify the daemon correctly handles the EEXIST race between recursive scan and queued inotify events. When IN_MASK_CREATE is available and a directory watch is added during recursive registration, the kernel may also deliver a queued IN_CREATE event for the same directory. The second inotify_add_watch() returns EEXIST, which must be treated as harmless. An earlier version of the listener crashed in this scenario. Reduce --start-timeout from the default 60 seconds to 10 seconds so that tests fail promptly when the daemon cannot start. Harden the test helpers to work in environments without procps (e.g., Fedora CI): fall back to reading /proc/$pid/stat for the process group ID when ps is unavailable, guard stop_git() against an empty pgid, and redirect stderr from kill to /dev/null to avoid noise when processes have already exited. Use set -m to enable job control in the submodule-pull test so that the background git pull gets its own process group, preventing the shell wait from blocking on the daemon. setsid() in the previous commit detaches the daemon itself, but the intermediate git pull process still needs its own process group for the test shell to manage it correctly. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- t/t7527-builtin-fsmonitor.sh | 89 +++++++++++++++++++++++++++++++++--- 1 file changed, 82 insertions(+), 7 deletions(-) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..774da5ac60 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -10,9 +10,58 @@ then test_done fi +# Verify that the filesystem delivers events to the daemon. +# On some configurations (e.g., overlayfs with older kernels), +# inotify watches succeed but events are never delivered. The +# cookie wait will time out and the daemon logs a trace message. +# +# Use "timeout" (if available) to guard each step against hangs. +maybe_timeout () { + if type timeout >/dev/null 2>&1 + then + timeout "$@" + else + shift + "$@" + fi +} +verify_fsmonitor_works () { + git init test_fsmonitor_smoke || return 1 + + GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && + export GIT_TRACE_FSMONITOR && + maybe_timeout 30 \ + git -C test_fsmonitor_smoke fsmonitor--daemon start \ + --start-timeout=10 + ret=$? + unset GIT_TRACE_FSMONITOR + if test $ret -ne 0 + then + rm -rf test_fsmonitor_smoke smoke.trace + return 1 + fi + + maybe_timeout 10 \ + test-tool -C test_fsmonitor_smoke fsmonitor-client query \ + --token 0 >/dev/null 2>&1 + maybe_timeout 5 \ + git -C test_fsmonitor_smoke fsmonitor--daemon stop 2>/dev/null + ! grep -q "cookie_wait timed out" "$PWD/smoke.trace" 2>/dev/null + ret=$? + rm -rf test_fsmonitor_smoke smoke.trace + return $ret +} + +if ! verify_fsmonitor_works +then + skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" + test_done +fi + stop_daemon_delete_repo () { r=$1 && - test_might_fail git -C $r fsmonitor--daemon stop && + test_might_fail maybe_timeout 30 \ + git -C $r fsmonitor--daemon stop 2>/dev/null rm -rf $1 } @@ -67,7 +116,7 @@ start_daemon () { export GIT_TEST_FSMONITOR_TOKEN fi && - git $r fsmonitor--daemon start && + git $r fsmonitor--daemon start --start-timeout=10 && git $r fsmonitor--daemon status ) } @@ -520,6 +569,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the @@ -910,7 +981,10 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" ' start_git_in_background () { git "$@" & git_pid=$! - git_pgid=$(ps -o pgid= -p $git_pid) + git_pgid=$(ps -o pgid= -p $git_pid 2>/dev/null || + awk '{print $5}' /proc/$git_pid/stat 2>/dev/null) && + git_pgid="${git_pgid## }" && + git_pgid="${git_pgid%% }" nr_tries_left=10 while true do @@ -921,15 +995,16 @@ start_git_in_background () { fi sleep 1 nr_tries_left=$(($nr_tries_left - 1)) - done >/dev/null 2>&1 & + done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! wait $git_pid } stop_git () { - while kill -0 -- -$git_pgid + test -n "$git_pgid" || return 0 + while kill -0 -- -$git_pgid 2>/dev/null do - kill -- -$git_pgid + kill -- -$git_pgid 2>/dev/null sleep 1 done } @@ -944,7 +1019,7 @@ stop_watchdog () { test_expect_success !MINGW "submodule implicitly starts daemon by pull" ' test_atexit "stop_watchdog" && - test_when_finished "stop_git; rm -rf cloned super sub" && + test_when_finished "set +m; stop_git; rm -rf cloned super sub" && create_super super && create_sub sub && -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v10 12/12] fsmonitor: convert shown khash to strset in do_handle_client 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (10 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 1:16 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Replace the khash-based string set used for deduplicating pathnames in do_handle_client() with a strset, which provides a cleaner interface for the same purpose. Since the paths are interned strings from the batch data, use strdup_strings=0 to avoid unnecessary copies. Suggested-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 299de2e4e2..b4b2a304e5 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -16,7 +16,7 @@ #include "fsmonitor--daemon.h" #include "simple-ipc.h" -#include "khash.h" +#include "strmap.h" #include "run-command.h" #include "trace.h" #include "trace2.h" @@ -674,8 +674,6 @@ static int fsmonitor_parse_client_token(const char *buf_token, return 0; } -KHASH_INIT(str, const char *, int, 0, kh_str_hash_func, kh_str_hash_equal) - static int do_handle_client(struct fsmonitor_daemon_state *state, const char *command, ipc_server_reply_cb *reply, @@ -692,8 +690,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown = NULL; - int hash_ret; + struct strset shown = STRSET_INIT; int do_trivial = 0; int do_flush = 0; int do_cookie = 0; @@ -882,14 +879,14 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, * so walk the batch list backwards from the current head back * to the batch (sequence number) they named. * - * We use khash to de-dup the list of pathnames. + * We use a strset to de-dup the list of pathnames. * * NEEDSWORK: each batch contains a list of interned strings, * so we only need to do pointer comparisons here to build the * hash table. Currently, we're still comparing the string * values. */ - shown = kh_init_str(); + strset_init_with_options(&shown, NULL, 0); for (batch = batch_head; batch && batch->batch_seq_nr > requested_oldest_seq_nr; batch = batch->next) { @@ -899,11 +896,9 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const char *s = batch->interned_paths[k]; size_t s_len; - if (kh_get_str(shown, s) != kh_end(shown)) + if (!strset_add(&shown, s)) duplicates++; else { - kh_put_str(shown, s, &hash_ret); - trace_printf_key(&trace_fsmonitor, "send[%"PRIuMAX"]: %s", count, s); @@ -973,7 +968,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: - kh_destroy_str(shown); + strset_clear(&shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (11 preceding siblings ...) 2026-03-05 1:16 ` [PATCH v10 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget ` (12 more replies) 12 siblings, 13 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan This series implements the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. It also fixes two memory leaks in the platform-independent daemon code and deduplicates the IPC and settings logic that is now shared between macOS and Linux. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. This series builds on work from https://github.com/git/git/pull/1352 by Eric DeCosta and https://github.com/git/git/pull/1667 by Marziyeh Esipreh, updated to work with the current codebase and address all review feedback. Changes since v10: * Reverted pre_exec_cb callback back to simple close_fd_above_stderr flag per Junio's clarification (same as v8) Changes since v9: * Fixed Windows build: close_fd_above_stderr() compiles as a no-op on Windows since there is no fork/exec Changes since v8: * Replaced close_fd_above_stderr flag with generic pre_exec_cb callback in struct child_process per Junio's review, with close_fd_above_stderr() as a ready-made callback Changes since v7: * Added patch 12: convert khash to strset in do_handle_client (Patrick's #leftoverbit suggestion) * Fixed "Forcing shutdown" trace message to start with lowercase * Fixed redundant statfs() call in find_mount() (caller already had the result) * Fixed CMakeLists.txt GIT-BUILD-OPTIONS: was hardcoded to "win32" for FSMONITOR_DAEMON_BACKEND and FSMONITOR_OS_SETTINGS, now uses the CMake variables * Fixed uninitialized strset on trivial response path (STRSET_INIT) * Removed V9FS_MAGIC from get_fs_typename() to match is_remote_fs() (9p is local VM mounts) * Split 30-second stop timeout into its own commit per review request * Fixed misleading indentation on shutdown assignment in handle_events() * Updated commit messages to describe all changes (test hardening, fsmonitor-ipc.c spawn changes) * Updated Makefile comment for FSMONITOR_OS_SETTINGS to mention fsm-ipc Changes since v6: * Introduced FSMONITOR_OS_SETTINGS build variable (set to "unix" for macOS and Linux, "win32" for Windows) to eliminate if/else conditionals in Makefile, meson.build, and CMakeLists.txt per Junio's review * Moved fsm-path-utils from FSMONITOR_OS_SETTINGS to FSMONITOR_DAEMON_BACKEND since path-utils files are platform-specific * Removed V9FS_MAGIC from remote filesystem detection (9p is used for local VM/container host mounts where fsmonitor works fine) * Removed redundant #include <libgen.h> (already provided by compat/posix.h) * Fixed cookie wait comment wording ("see" to "observe") * Rewrote commit messages for IPC and settings dedup patches Changes since v5: * Split monolithic commit into 10-patch series per Patrick's review * Deduplicated fsm-ipc and fsm-settings into shared Unix implementations * Rewrote commit message with prose paragraphs, explain inotify vs fanotify, removed "Issues addressed" sections, added Based-on-patch-by trailers * Removed redundant includes already provided by compat/posix.h * Fixed error/trace message capitalization per coding guidelines * Fixed stale rename check interval from 1000 seconds to 1 second * Changed poll timeout from 1ms to 50ms to reduce idle CPU wake-ups * Replaced infinite pthread_cond_wait cookie loop with one-second pthread_cond_timedwait (prevents daemon hangs on overlay filesystems where events are never delivered) * Added pthread_cond_timedwait to Windows pthread compatibility layer * Separated test into its own commit with smoke test that skips when inotify events are not delivered (e.g., overlayfs with older kernels) * Fixed test hang on Fedora CI: stop_git() looped forever when ps was unavailable because bash in POSIX/sh mode returns exit 0 from kill with an empty process group argument. Fixed by falling back to /proc/$pid/stat for process group ID and guarding stop_git against empty pgid. * Redirect spawn_daemon() stdout/stderr to /dev/null and close inherited file descriptors to prevent the intermediate process from holding test pipe file descriptors * Call setsid() on daemon detach to prevent shells with job control from waiting on the daemon process group * Close inherited file descriptors 3-7 in the test watchdog subprocess * Added 30-second timeout to "fsmonitor--daemon stop" to prevent indefinite blocking * Added helpful error message when inotify watch limit (max_user_watches) is reached * Initialize fd_inotify to -1 and use fd >= 0 check for correct fd 0 handling * Use sysconf(_SC_OPEN_MAX) instead of hardcoded 1024 for fd close limit * Check setsid() return value Changes since v4: * Added Meson build support Changes since v3: * Fix crash on rapid nested directory creation (EEXIST from inotify_add_watch with IN_MASK_CREATE) * Extensive stress testing Changes since v2: * Fix khash memory leak in do_handle_client Changes since v1: * Fix hashmap memory leak in fsmonitor_run_daemon() Paul Tarjan (12): fsmonitor: fix khash memory leak in do_handle_client fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon compat/win32: add pthread_cond_timedwait fsmonitor: use pthread_cond_timedwait for cookie wait fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c fsmonitor: implement filesystem change listener for Linux run-command: add close_fd_above_stderr option fsmonitor: close inherited file descriptors and detach in daemon fsmonitor: add timeout to daemon stop command fsmonitor: add tests for Linux fsmonitor: convert shown khash to strset in do_handle_client Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- Makefile | 6 +- builtin/fsmonitor--daemon.c | 92 ++- compat/fsmonitor/fsm-health-linux.c | 33 + .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 +++++ ...-settings-darwin.c => fsm-settings-unix.c} | 0 compat/win32/pthread.c | 26 + compat/win32/pthread.h | 2 + config.mak.uname | 12 +- contrib/buildsystems/CMakeLists.txt | 33 +- fsmonitor-ipc.c | 3 + meson.build | 13 +- run-command.c | 12 + run-command.h | 9 + t/t7527-builtin-fsmonitor.sh | 89 ++- 18 files changed, 1262 insertions(+), 63 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) base-commit: 3e0db84c88c57e70ac8be8c196dfa92c5d656fbc Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2147%2Fptarjan%2Fclaude%2Fupdate-pr-1352-current-85Gk8-v11 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2147/ptarjan/claude/update-pr-1352-current-85Gk8-v11 Pull-Request: https://github.com/git/git/pull/2147 Range-diff vs v10: 1: 4d4dec8fa1 = 1: 4d4dec8fa1 fsmonitor: fix khash memory leak in do_handle_client 2: cb270120f0 = 2: cb270120f0 fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 3: 44a063074d = 3: 44a063074d compat/win32: add pthread_cond_timedwait 4: b1081d1e13 = 4: b1081d1e13 fsmonitor: use pthread_cond_timedwait for cookie wait 5: dec0fb144f = 5: dec0fb144f fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c 6: b2aaadb4ae = 6: b2aaadb4ae fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c 7: 03cf12d01b = 7: 03cf12d01b fsmonitor: implement filesystem change listener for Linux 8: 39dcfbb7c8 ! 8: 50f5b4676e run-command: add pre-exec callback for child processes @@ Metadata Author: Paul Tarjan <github@paulisageek.com> ## Commit message ## - run-command: add pre-exec callback for child processes + run-command: add close_fd_above_stderr option - Add a pre_exec_cb function pointer to struct child_process that is - invoked in the child between fork and exec. This gives callers a - place to perform setup that must happen in the child's context, - such as closing inherited file descriptors. - - Provide close_fd_above_stderr() as a ready-made callback that - closes file descriptors 3 and above (skipping the child-notifier - pipe), capped at sysconf(_SC_OPEN_MAX) or 4096, whichever is - smaller. + Add a close_fd_above_stderr flag to struct child_process. When set, + the child closes file descriptors 3 and above between fork and exec + (skipping the child-notifier pipe), capped at sysconf(_SC_OPEN_MAX) + or 4096, whichever is smaller. This prevents the child from + inheriting pipe endpoints or other descriptors from the parent + environment (e.g., the test harness). Signed-off-by: Paul Tarjan <github@paulisageek.com> @@ run-command.c: static void atfork_parent(struct atfork_state *as) + #endif /* GIT_WINDOWS_NATIVE */ -+void close_fd_above_stderr(void) -+{ -+#ifndef GIT_WINDOWS_NATIVE -+ long max_fd = sysconf(_SC_OPEN_MAX); -+ int fd; -+ if (max_fd < 0 || max_fd > 4096) -+ max_fd = 4096; -+ for (fd = 3; fd < max_fd; fd++) { -+ if (fd != child_notifier) -+ close(fd); -+ } -+#endif -+} -+ static inline void set_cloexec(int fd) - { - int flags = fcntl(fd, F_GETFD); @@ run-command.c: fail_pipe: child_close(cmd->out); } -+ if (cmd->pre_exec_cb) -+ cmd->pre_exec_cb(); ++ if (cmd->close_fd_above_stderr) { ++ long max_fd = sysconf(_SC_OPEN_MAX); ++ int fd; ++ if (max_fd < 0 || max_fd > 4096) ++ max_fd = 4096; ++ for (fd = 3; fd < max_fd; fd++) { ++ if (fd != child_notifier) ++ close(fd); ++ } ++ } + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); @@ run-command.h: struct child_process { unsigned wait_after_clean:1; + + /** -+ * If set, the callback is invoked in the child between fork and -+ * exec. It can be used, for example, to close inherited file -+ * descriptors that the child should not keep open. ++ * Close file descriptors 3 and above in the child after forking ++ * but before exec. This prevents the child from inheriting ++ * pipe endpoints or other descriptors from the parent ++ * environment (e.g., the test harness). + */ -+ void (*pre_exec_cb)(void); ++ unsigned close_fd_above_stderr:1; + void (*clean_on_exit_handler)(struct child_process *process); }; -@@ run-command.h: struct child_process { - .env = STRVEC_INIT, \ - } - -+/** -+ * Close file descriptors 3 and above. Suitable for use as a -+ * pre_exec_cb to prevent the child from inheriting pipe endpoints -+ * or other descriptors from the parent environment. On Windows -+ * this is a no-op since there is no fork/exec. -+ */ -+void close_fd_above_stderr(void); -+ - /** - * The functions: start_command, finish_command, run_command do the following: - * 9: 5db0591c15 ! 9: 057b3098bc fsmonitor: close inherited file descriptors and detach in daemon @@ builtin/fsmonitor--daemon.c: static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; -+ cp.pre_exec_cb = close_fd_above_stderr; ++ cp.close_fd_above_stderr = 1; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); @@ fsmonitor-ipc.c: static int spawn_daemon(void) cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; -+ cmd.pre_exec_cb = close_fd_above_stderr; ++ cmd.close_fd_above_stderr = 1; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); 10: 8a9a6ba4fa = 10: e6bc3bfcb2 fsmonitor: add timeout to daemon stop command 11: 27d5560007 = 11: 81f8cd1599 fsmonitor: add tests for Linux 12: 8ea20aab4c = 12: 8fa6a74e0d fsmonitor: convert shown khash to strset in do_handle_client -- gitgitgadget ^ permalink raw reply [flat|nested] 129+ messages in thread
* [PATCH v11 01/12] fsmonitor: fix khash memory leak in do_handle_client 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget ` (11 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `shown` kh_str_t was freed with kh_release_str() at a point in the code only reachable in the non-trivial response path. When the client receives a trivial response, the code jumps to the `cleanup` label, skipping the kh_release_str() call entirely and leaking the hash table. Fix this by initializing `shown` to NULL and moving the cleanup to the `cleanup` label using kh_destroy_str(), which is safe to call on NULL. This ensures the hash table is freed regardless of which code path is taken. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 242c594646..bc4571938c 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -671,7 +671,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown; + kh_str_t *shown = NULL; int hash_ret; int do_trivial = 0; int do_flush = 0; @@ -909,8 +909,6 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, total_response_len += payload.len; } - kh_release_str(shown); - pthread_mutex_lock(&state->main_lock); if (token_data->client_ref_count > 0) @@ -954,6 +952,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: + kh_destroy_str(shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget ` (10 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The `state.cookies` hashmap is initialized during daemon startup but never freed during cleanup in the `done:` label of fsmonitor_run_daemon(). The cookie entries also have names allocated via strbuf_detach() that must be freed individually. Iterate the hashmap to free each cookie name, then call hashmap_clear_and_free() to release the entries and table. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index bc4571938c..d8d32b01ef 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1404,6 +1404,15 @@ static int fsmonitor_run_daemon(void) done: pthread_cond_destroy(&state.cookies_cond); pthread_mutex_destroy(&state.main_lock); + { + struct hashmap_iter iter; + struct fsmonitor_cookie_item *cookie; + + hashmap_for_each_entry(&state.cookies, &iter, cookie, entry) + free(cookie->name); + hashmap_clear_and_free(&state.cookies, + struct fsmonitor_cookie_item, entry); + } fsm_listen__dtor(&state); fsm_health__dtor(&state); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 03/12] compat/win32: add pthread_cond_timedwait 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget ` (9 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a pthread_cond_timedwait() implementation to the Windows pthread compatibility layer using SleepConditionVariableCS() with a millisecond timeout computed from the absolute deadline. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/win32/pthread.c | 26 ++++++++++++++++++++++++++ compat/win32/pthread.h | 2 ++ 2 files changed, 28 insertions(+) diff --git a/compat/win32/pthread.c b/compat/win32/pthread.c index 7e93146963..398caa9602 100644 --- a/compat/win32/pthread.c +++ b/compat/win32/pthread.c @@ -66,3 +66,29 @@ int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex) return err_win_to_posix(GetLastError()); return 0; } + +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime) +{ + struct timeval now; + long long now_ms, deadline_ms; + DWORD timeout_ms; + + gettimeofday(&now, NULL); + now_ms = (long long)now.tv_sec * 1000 + now.tv_usec / 1000; + deadline_ms = (long long)abstime->tv_sec * 1000 + + abstime->tv_nsec / 1000000; + + if (deadline_ms <= now_ms) + return ETIMEDOUT; + else + timeout_ms = (DWORD)(deadline_ms - now_ms); + + if (SleepConditionVariableCS(cond, mutex, timeout_ms) == 0) { + DWORD err = GetLastError(); + if (err == ERROR_TIMEOUT) + return ETIMEDOUT; + return err_win_to_posix(err); + } + return 0; +} diff --git a/compat/win32/pthread.h b/compat/win32/pthread.h index ccacc5a53b..d80df8d12a 100644 --- a/compat/win32/pthread.h +++ b/compat/win32/pthread.h @@ -64,6 +64,8 @@ int win32_pthread_join(pthread_t *thread, void **value_ptr); pthread_t pthread_self(void); int pthread_cond_wait(pthread_cond_t *cond, pthread_mutex_t *mutex); +int pthread_cond_timedwait(pthread_cond_t *cond, pthread_mutex_t *mutex, + const struct timespec *abstime); static inline void NORETURN pthread_exit(void *ret) { -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (2 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget ` (8 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The cookie wait in with_lock__wait_for_cookie() uses an infinite pthread_cond_wait() loop. The existing comment notes the desire to switch to pthread_cond_timedwait(), but the routine was not available in git thread-utils. On certain container or overlay filesystems, inotify watches may succeed but events are never delivered. In this case the daemon would hang indefinitely waiting for the cookie event, which in turn causes the client to hang. Replace the infinite wait with a one-second timeout using pthread_cond_timedwait(). If the timeout fires, report an error and let the client proceed with a trivial (full-scan) response rather than blocking forever. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index d8d32b01ef..c8ec7b722e 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -197,20 +197,31 @@ static enum fsmonitor_cookie_item_result with_lock__wait_for_cookie( unlink(cookie_pathname.buf); /* - * Technically, this is an infinite wait (well, unless another - * thread sends us an abort). I'd like to change this to - * use `pthread_cond_timedwait()` and return an error/timeout - * and let the caller do the trivial response thing, but we - * don't have that routine in our thread-utils. - * - * After extensive beta testing I'm not really worried about - * this. Also note that the above open() and unlink() calls - * will cause at least two FS events on that path, so the odds - * of getting stuck are pretty slim. + * Wait for the listener thread to observe the cookie file. + * Time out after a short interval so that the client + * does not hang forever if the filesystem does not deliver + * events (e.g., on certain container/overlay filesystems + * where inotify watches succeed but events never arrive). */ - while (cookie->result == FCIR_INIT) - pthread_cond_wait(&state->cookies_cond, - &state->main_lock); + { + struct timeval now; + struct timespec ts; + int err = 0; + + gettimeofday(&now, NULL); + ts.tv_sec = now.tv_sec + 1; + ts.tv_nsec = now.tv_usec * 1000; + + while (cookie->result == FCIR_INIT && !err) + err = pthread_cond_timedwait(&state->cookies_cond, + &state->main_lock, + &ts); + if (err == ETIMEDOUT && cookie->result == FCIR_INIT) { + trace_printf_key(&trace_fsmonitor, + "cookie_wait timed out"); + cookie->result = FCIR_ERROR; + } + } done: hashmap_remove(&state->cookies, &cookie->entry, NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (3 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget ` (7 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor IPC path logic in fsm-ipc-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-ipc-unix.c to reflect that it is shared by all Unix platforms. Introduce FSMONITOR_OS_SETTINGS (set to "unix" for non-Windows, "win32" for Windows) as a separate variable from FSMONITOR_DAEMON_BACKEND so that the build files can distinguish between platform-specific files (listen, health, path-utils) and shared Unix files (ipc, settings). Move fsm-ipc to the FSMONITOR_OS_SETTINGS section in the Makefile, and switch fsm-path-utils to use FSMONITOR_DAEMON_BACKEND since path-utils is platform-specific (there will be separate darwin and linux versions). Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Makefile | 6 ++--- .../{fsm-ipc-darwin.c => fsm-ipc-unix.c} | 0 config.mak.uname | 2 +- contrib/buildsystems/CMakeLists.txt | 25 +++++++++---------- meson.build | 7 ++++-- 5 files changed, 21 insertions(+), 19 deletions(-) rename compat/fsmonitor/{fsm-ipc-darwin.c => fsm-ipc-unix.c} (100%) diff --git a/Makefile b/Makefile index 8aa489f3b6..080d009bf0 100644 --- a/Makefile +++ b/Makefile @@ -417,7 +417,7 @@ include shared.mak # If your platform has OS-specific ways to tell if a repo is incompatible with # fsmonitor (whether the hook or IPC daemon version), set FSMONITOR_OS_SETTINGS # to the "<name>" of the corresponding `compat/fsmonitor/fsm-settings-<name>.c` -# that implements the `fsm_os_settings__*()` routines. +# and `compat/fsmonitor/fsm-ipc-<name>.c` files. # # Define LINK_FUZZ_PROGRAMS if you want `make all` to also build the fuzz test # programs in oss-fuzz/. @@ -2365,13 +2365,13 @@ ifdef FSMONITOR_DAEMON_BACKEND COMPAT_CFLAGS += -DHAVE_FSMONITOR_DAEMON_BACKEND COMPAT_OBJS += compat/fsmonitor/fsm-listen-$(FSMONITOR_DAEMON_BACKEND).o COMPAT_OBJS += compat/fsmonitor/fsm-health-$(FSMONITOR_DAEMON_BACKEND).o - COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef FSMONITOR_OS_SETTINGS COMPAT_CFLAGS += -DHAVE_FSMONITOR_OS_SETTINGS + COMPAT_OBJS += compat/fsmonitor/fsm-ipc-$(FSMONITOR_OS_SETTINGS).o COMPAT_OBJS += compat/fsmonitor/fsm-settings-$(FSMONITOR_OS_SETTINGS).o - COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_OS_SETTINGS).o + COMPAT_OBJS += compat/fsmonitor/fsm-path-utils-$(FSMONITOR_DAEMON_BACKEND).o endif ifdef WITH_BREAKING_CHANGES diff --git a/compat/fsmonitor/fsm-ipc-darwin.c b/compat/fsmonitor/fsm-ipc-unix.c similarity index 100% rename from compat/fsmonitor/fsm-ipc-darwin.c rename to compat/fsmonitor/fsm-ipc-unix.c diff --git a/config.mak.uname b/config.mak.uname index 3c35ae33a3..33877020e9 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -165,7 +165,7 @@ ifeq ($(uname_S),Darwin) ifndef NO_PTHREADS ifndef NO_UNIX_SOCKETS FSMONITOR_DAEMON_BACKEND = darwin - FSMONITOR_OS_SETTINGS = darwin + FSMONITOR_OS_SETTINGS = unix endif endif diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 28877feb9d..6197d5729c 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -291,23 +291,22 @@ endif() if(SUPPORTS_SIMPLE_IPC) if(CMAKE_SYSTEM_NAME STREQUAL "Windows") - add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-win32.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-win32.c) - - add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-win32.c) + set(FSMONITOR_DAEMON_BACKEND "win32") + set(FSMONITOR_OS_SETTINGS "win32") elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") + set(FSMONITOR_DAEMON_BACKEND "darwin") + set(FSMONITOR_OS_SETTINGS "unix") + endif() + + if(FSMONITOR_DAEMON_BACKEND) add_compile_definitions(HAVE_FSMONITOR_DAEMON_BACKEND) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-darwin.c) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-listen-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-health-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-ipc-${FSMONITOR_OS_SETTINGS}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-darwin.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) endif() endif() diff --git a/meson.build b/meson.build index dd52efd1c8..86a68365a9 100644 --- a/meson.build +++ b/meson.build @@ -1320,10 +1320,13 @@ else endif fsmonitor_backend = '' +fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' + fsmonitor_os = 'win32' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' + fsmonitor_os = 'unix' libgit_dependencies += dependency('CoreServices') endif if fsmonitor_backend != '' @@ -1332,14 +1335,14 @@ if fsmonitor_backend != '' libgit_sources += [ 'compat/fsmonitor/fsm-health-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-ipc-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_backend) +build_options_config.set_quoted('FSMONITOR_OS_SETTINGS', fsmonitor_os) if not get_option('b_sanitize').contains('address') and get_option('regex').allowed() and compiler.has_header('regex.h') and compiler.get_define('REG_STARTEND', prefix: '#include <regex.h>') != '' build_options_config.set('NO_REGEX', '') -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (4 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The fsmonitor settings logic in fsm-settings-darwin.c is not Darwin-specific and will be reused by the upcoming Linux implementation. Rename it to fsm-settings-unix.c to reflect that it is shared by all Unix platforms. Update the build files (meson.build and CMakeLists.txt) to use FSMONITOR_OS_SETTINGS for fsm-settings, matching the approach already used for fsm-ipc. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} | 0 contrib/buildsystems/CMakeLists.txt | 2 +- meson.build | 2 +- 3 files changed, 2 insertions(+), 2 deletions(-) rename compat/fsmonitor/{fsm-settings-darwin.c => fsm-settings-unix.c} (100%) diff --git a/compat/fsmonitor/fsm-settings-darwin.c b/compat/fsmonitor/fsm-settings-unix.c similarity index 100% rename from compat/fsmonitor/fsm-settings-darwin.c rename to compat/fsmonitor/fsm-settings-unix.c diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index 6197d5729c..d613809e26 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -306,7 +306,7 @@ if(SUPPORTS_SIMPLE_IPC) list(APPEND compat_SOURCES compat/fsmonitor/fsm-path-utils-${FSMONITOR_DAEMON_BACKEND}.c) add_compile_definitions(HAVE_FSMONITOR_OS_SETTINGS) - list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_DAEMON_BACKEND}.c) + list(APPEND compat_SOURCES compat/fsmonitor/fsm-settings-${FSMONITOR_OS_SETTINGS}.c) endif() endif() diff --git a/meson.build b/meson.build index 86a68365a9..4f0c0a33b8 100644 --- a/meson.build +++ b/meson.build @@ -1338,7 +1338,7 @@ if fsmonitor_backend != '' 'compat/fsmonitor/fsm-ipc-' + fsmonitor_os + '.c', 'compat/fsmonitor/fsm-listen-' + fsmonitor_backend + '.c', 'compat/fsmonitor/fsm-path-utils-' + fsmonitor_backend + '.c', - 'compat/fsmonitor/fsm-settings-' + fsmonitor_backend + '.c', + 'compat/fsmonitor/fsm-settings-' + fsmonitor_os + '.c', ] endif build_options_config.set_quoted('FSMONITOR_DAEMON_BACKEND', fsmonitor_backend) -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 07/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (5 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 08/12] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget ` (5 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Implement the built-in fsmonitor daemon for Linux using the inotify API, bringing it to feature parity with the existing Windows and macOS implementations. The implementation uses inotify rather than fanotify because fanotify requires either CAP_SYS_ADMIN or CAP_PERFMON capabilities, making it unsuitable for an unprivileged user-space daemon. While inotify has the limitation of requiring a separate watch on every directory (unlike macOS's FSEvents, which can monitor an entire directory tree with a single watch), it operates without elevated privileges and provides the per-file event granularity needed for fsmonitor. The listener uses inotify_init1(O_NONBLOCK) with a poll loop that checks for events with a 50-millisecond timeout, keeping the inotify queue well-drained to minimize the risk of overflows. Bidirectional hashmaps map between watch descriptors and directory paths for efficient event resolution. Directory renames are tracked using inotify's cookie mechanism to correlate IN_MOVED_FROM and IN_MOVED_TO event pairs; a periodic check detects stale renames where the matching IN_MOVED_TO never arrived, forcing a resync. New directory creation triggers recursive watch registration to ensure all subdirectories are monitored. The IN_MASK_CREATE flag is used where available to prevent modifying existing watches, with a fallback for older kernels. When IN_MASK_CREATE is available and inotify_add_watch returns EEXIST, it means another thread or recursive scan has already registered the watch, so it is safe to ignore. Remote filesystem detection uses statfs() to identify network-mounted filesystems (NFS, CIFS, SMB, FUSE, etc.) via their magic numbers. Mount point information is read from /proc/mounts and matched against the statfs f_fsid to get accurate, human-readable filesystem type names for logging. When the .git directory is on a remote filesystem, the IPC socket falls back to $HOME or a user-configured directory via the fsmonitor.socketDir setting. Based-on-patch-by: Eric DeCosta <edecosta@mathworks.com> Based-on-patch-by: Marziyeh Esipreh <marziyeh.esipreh@gmail.com> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- Documentation/config/fsmonitor--daemon.adoc | 4 +- Documentation/git-fsmonitor--daemon.adoc | 28 +- compat/fsmonitor/fsm-health-linux.c | 33 + compat/fsmonitor/fsm-listen-linux.c | 746 ++++++++++++++++++++ compat/fsmonitor/fsm-path-utils-linux.c | 217 ++++++ config.mak.uname | 10 + contrib/buildsystems/CMakeLists.txt | 8 +- meson.build | 4 + 8 files changed, 1042 insertions(+), 8 deletions(-) create mode 100644 compat/fsmonitor/fsm-health-linux.c create mode 100644 compat/fsmonitor/fsm-listen-linux.c create mode 100644 compat/fsmonitor/fsm-path-utils-linux.c diff --git a/Documentation/config/fsmonitor--daemon.adoc b/Documentation/config/fsmonitor--daemon.adoc index 671f9b9462..6f8386e291 100644 --- a/Documentation/config/fsmonitor--daemon.adoc +++ b/Documentation/config/fsmonitor--daemon.adoc @@ -4,8 +4,8 @@ fsmonitor.allowRemote:: behavior. Only respected when `core.fsmonitor` is set to `true`. fsmonitor.socketDir:: - This Mac OS-specific option, if set, specifies the directory in + This Mac OS and Linux-specific option, if set, specifies the directory in which to create the Unix domain socket used for communication between the fsmonitor daemon and various Git commands. The directory must - reside on a native Mac OS filesystem. Only respected when `core.fsmonitor` + reside on a native filesystem. Only respected when `core.fsmonitor` is set to `true`. diff --git a/Documentation/git-fsmonitor--daemon.adoc b/Documentation/git-fsmonitor--daemon.adoc index 8fe5241b08..12fa866a64 100644 --- a/Documentation/git-fsmonitor--daemon.adoc +++ b/Documentation/git-fsmonitor--daemon.adoc @@ -76,9 +76,9 @@ repositories; this may be overridden by setting `fsmonitor.allowRemote` to correctly with all network-mounted repositories, so such use is considered experimental. -On Mac OS, the inter-process communication (IPC) between various Git +On Mac OS and Linux, the inter-process communication (IPC) between various Git commands and the fsmonitor daemon is done via a Unix domain socket (UDS) -- a -special type of file -- which is supported by native Mac OS filesystems, +special type of file -- which is supported by native Mac OS and Linux filesystems, but not on network-mounted filesystems, NTFS, or FAT32. Other filesystems may or may not have the needed support; the fsmonitor daemon is not guaranteed to work with these filesystems and such use is considered experimental. @@ -87,13 +87,33 @@ By default, the socket is created in the `.git` directory. However, if the `.git` directory is on a network-mounted filesystem, it will instead be created at `$HOME/.git-fsmonitor-*` unless `$HOME` itself is on a network-mounted filesystem, in which case you must set the configuration -variable `fsmonitor.socketDir` to the path of a directory on a Mac OS native +variable `fsmonitor.socketDir` to the path of a directory on a native filesystem in which to create the socket file. If none of the above directories (`.git`, `$HOME`, or `fsmonitor.socketDir`) -is on a native Mac OS file filesystem the fsmonitor daemon will report an +is on a native filesystem the fsmonitor daemon will report an error that will cause the daemon and the currently running command to exit. +LINUX CAVEATS +~~~~~~~~~~~~~ + +On Linux, the fsmonitor daemon uses inotify to monitor filesystem events. +The inotify system has per-user limits on the number of watches that can +be created. The default limit is typically 8192 watches per user. + +For large repositories with many directories, you may need to increase +this limit. Check the current limit with: + + cat /proc/sys/fs/inotify/max_user_watches + +To temporarily increase the limit: + + sudo sysctl fs.inotify.max_user_watches=65536 + +To make the change permanent, add to `/etc/sysctl.conf`: + + fs.inotify.max_user_watches=65536 + CONFIGURATION ------------- diff --git a/compat/fsmonitor/fsm-health-linux.c b/compat/fsmonitor/fsm-health-linux.c new file mode 100644 index 0000000000..43d67c4b8b --- /dev/null +++ b/compat/fsmonitor/fsm-health-linux.c @@ -0,0 +1,33 @@ +#include "git-compat-util.h" +#include "config.h" +#include "fsmonitor-ll.h" +#include "fsm-health.h" +#include "fsmonitor--daemon.h" + +/* + * The Linux fsmonitor implementation uses inotify which has its own + * mechanisms for detecting filesystem unmount and other events that + * would require the daemon to shutdown. Therefore, we don't need + * a separate health thread like Windows does. + * + * These stub functions satisfy the interface requirements. + */ + +int fsm_health__ctor(struct fsmonitor_daemon_state *state UNUSED) +{ + return 0; +} + +void fsm_health__dtor(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__loop(struct fsmonitor_daemon_state *state UNUSED) +{ + return; +} + +void fsm_health__stop_async(struct fsmonitor_daemon_state *state UNUSED) +{ +} diff --git a/compat/fsmonitor/fsm-listen-linux.c b/compat/fsmonitor/fsm-listen-linux.c new file mode 100644 index 0000000000..e3dca14b62 --- /dev/null +++ b/compat/fsmonitor/fsm-listen-linux.c @@ -0,0 +1,746 @@ +#include "git-compat-util.h" +#include "dir.h" +#include "fsmonitor-ll.h" +#include "fsm-listen.h" +#include "fsmonitor--daemon.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "simple-ipc.h" +#include "string-list.h" +#include "trace.h" + +#include <sys/inotify.h> + +/* + * Safe value to bitwise OR with rest of mask for + * kernels that do not support IN_MASK_CREATE + */ +#ifndef IN_MASK_CREATE +#define IN_MASK_CREATE 0x00000000 +#endif + +enum shutdown_reason { + SHUTDOWN_CONTINUE = 0, + SHUTDOWN_STOP, + SHUTDOWN_ERROR, + SHUTDOWN_FORCE +}; + +struct watch_entry { + struct hashmap_entry ent; + int wd; + uint32_t cookie; + const char *dir; +}; + +struct rename_entry { + struct hashmap_entry ent; + time_t whence; + uint32_t cookie; + const char *dir; +}; + +struct fsm_listen_data { + int fd_inotify; + enum shutdown_reason shutdown; + struct hashmap watches; + struct hashmap renames; + struct hashmap revwatches; +}; + +static int watch_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return e1->wd != e2->wd; +} + +static int revwatches_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct watch_entry *e1, *e2; + + e1 = container_of(eptr, const struct watch_entry, ent); + e2 = container_of(entry_or_key, const struct watch_entry, ent); + return strcmp(e1->dir, e2->dir); +} + +static int rename_entry_cmp(const void *cmp_data UNUSED, + const struct hashmap_entry *eptr, + const struct hashmap_entry *entry_or_key, + const void *keydata UNUSED) +{ + const struct rename_entry *e1, *e2; + + e1 = container_of(eptr, const struct rename_entry, ent); + e2 = container_of(entry_or_key, const struct rename_entry, ent); + return e1->cookie != e2->cookie; +} + +/* + * Register an inotify watch, add watch descriptor to path mapping + * and the reverse mapping. + */ +static int add_watch(const char *path, struct fsm_listen_data *data) +{ + const char *interned = strintern(path); + struct watch_entry *w1, *w2; + + /* add the inotify watch, don't allow watches to be modified */ + int wd = inotify_add_watch(data->fd_inotify, interned, + (IN_ALL_EVENTS | IN_ONLYDIR | IN_MASK_CREATE) + ^ IN_ACCESS ^ IN_CLOSE ^ IN_OPEN); + if (wd < 0) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted or is not a directory */ + if (errno == EEXIST) + return 0; /* watch already exists, no action needed */ + if (errno == ENOSPC) + return error(_("inotify watch limit reached; " + "increase fs.inotify.max_user_watches")); + return error_errno(_("inotify_add_watch('%s') failed"), interned); + } + + /* add watch descriptor -> directory mapping */ + CALLOC_ARRAY(w1, 1); + w1->wd = wd; + w1->dir = interned; + hashmap_entry_init(&w1->ent, memhash(&w1->wd, sizeof(int))); + hashmap_add(&data->watches, &w1->ent); + + /* add directory -> watch descriptor mapping */ + CALLOC_ARRAY(w2, 1); + w2->wd = wd; + w2->dir = interned; + hashmap_entry_init(&w2->ent, strhash(w2->dir)); + hashmap_add(&data->revwatches, &w2->ent); + + return 0; +} + +/* + * Remove the inotify watch, the watch descriptor to path mapping + * and the reverse mapping. + */ +static void remove_watch(struct watch_entry *w, struct fsm_listen_data *data) +{ + struct watch_entry k1, k2, *w1, *w2; + + /* remove watch, ignore error if kernel already did it */ + if (inotify_rm_watch(data->fd_inotify, w->wd) && errno != EINVAL) + error_errno(_("inotify_rm_watch() failed")); + + k1.wd = w->wd; + hashmap_entry_init(&k1.ent, memhash(&k1.wd, sizeof(int))); + w1 = hashmap_remove_entry(&data->watches, &k1, ent, NULL); + if (!w1) + BUG("double remove of watch for '%s'", w->dir); + + if (w1->cookie) + BUG("removing watch for '%s' which has a pending rename", w1->dir); + + k2.dir = w->dir; + hashmap_entry_init(&k2.ent, strhash(k2.dir)); + w2 = hashmap_remove_entry(&data->revwatches, &k2, ent, NULL); + if (!w2) + BUG("double remove of reverse watch for '%s'", w->dir); + + /* w1->dir and w2->dir are interned strings, we don't own them */ + free(w1); + free(w2); +} + +/* + * Check for stale directory renames. + * + * https://man7.org/linux/man-pages/man7/inotify.7.html + * + * Allow for some small timeout to account for the fact that insertion of the + * IN_MOVED_FROM+IN_MOVED_TO event pair is not atomic, and the possibility that + * there may not be any IN_MOVED_TO event. + * + * If the IN_MOVED_TO event is not received within the timeout then events have + * been missed and the monitor is in an inconsistent state with respect to the + * filesystem. + */ +static int check_stale_dir_renames(struct hashmap *renames, time_t max_age) +{ + struct rename_entry *re; + struct hashmap_iter iter; + + hashmap_for_each_entry(renames, &iter, re, ent) { + if (re->whence <= max_age) + return -1; + } + return 0; +} + +/* + * Track pending renames. + * + * Tracking is done via an event cookie to watch descriptor mapping. + * + * A rename is not complete until matching an IN_MOVED_TO event is received + * for a corresponding IN_MOVED_FROM event. + */ +static void add_dir_rename(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct watch_entry k, *w; + struct rename_entry *re; + + /* lookup the watch descriptor for the given path */ + k.dir = path; + hashmap_entry_init(&k.ent, strhash(path)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (!w) { + /* + * This can happen in rare cases where the directory was + * moved before we had a chance to add a watch on it. + * Just ignore this rename. + */ + trace_printf_key(&trace_fsmonitor, + "no watch found for rename from '%s'", path); + return; + } + w->cookie = cookie; + + /* add the pending rename to match against later */ + CALLOC_ARRAY(re, 1); + re->dir = w->dir; + re->cookie = w->cookie; + re->whence = time(NULL); + hashmap_entry_init(&re->ent, memhash(&re->cookie, sizeof(uint32_t))); + hashmap_add(&data->renames, &re->ent); +} + +/* + * Handle directory renames + * + * Once an IN_MOVED_TO event is received, lookup the rename tracking information + * via the event cookie and use this information to update the watch. + */ +static void rename_dir(uint32_t cookie, const char *path, + struct fsm_listen_data *data) +{ + struct rename_entry rek, *re; + struct watch_entry k, *w; + + /* lookup a pending rename to match */ + rek.cookie = cookie; + hashmap_entry_init(&rek.ent, memhash(&rek.cookie, sizeof(uint32_t))); + re = hashmap_get_entry(&data->renames, &rek, ent, NULL); + if (re) { + k.dir = re->dir; + hashmap_entry_init(&k.ent, strhash(k.dir)); + w = hashmap_get_entry(&data->revwatches, &k, ent, NULL); + if (w) { + w->cookie = 0; /* rename handled */ + remove_watch(w, data); + if (add_watch(path, data)) + trace_printf_key(&trace_fsmonitor, + "failed to add watch for renamed dir '%s'", + path); + } else { + /* Directory was moved out of watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching watch for rename to '%s'", path); + } + hashmap_remove_entry(&data->renames, &rek, ent, NULL); + free(re); + } else { + /* Directory was moved from outside the watch tree */ + trace_printf_key(&trace_fsmonitor, + "no matching cookie for rename to '%s'", path); + } +} + +/* + * Recursively add watches to every directory under path + */ +static int register_inotify(const char *path, + struct fsmonitor_daemon_state *state, + struct fsmonitor_batch *batch) +{ + DIR *dir; + const char *rel; + struct strbuf current = STRBUF_INIT; + struct dirent *de; + struct stat fs; + int ret = -1; + + dir = opendir(path); + if (!dir) { + if (errno == ENOENT || errno == ENOTDIR) + return 0; /* directory was deleted */ + return error_errno(_("opendir('%s') failed"), path); + } + + while ((de = readdir_skip_dot_and_dotdot(dir)) != NULL) { + strbuf_reset(¤t); + strbuf_addf(¤t, "%s/%s", path, de->d_name); + if (lstat(current.buf, &fs)) { + if (errno == ENOENT) + continue; /* file was deleted */ + error_errno(_("lstat('%s') failed"), current.buf); + goto failed; + } + + /* recurse into directory */ + if (S_ISDIR(fs.st_mode)) { + if (add_watch(current.buf, state->listen_data)) + goto failed; + if (register_inotify(current.buf, state, batch)) + goto failed; + } else if (batch) { + rel = current.buf + state->path_worktree_watch.len + 1; + trace_printf_key(&trace_fsmonitor, "explicitly adding '%s'", rel); + fsmonitor_batch__add_path(batch, rel); + } + } + ret = 0; + +failed: + strbuf_release(¤t); + if (closedir(dir) < 0) + return error_errno(_("closedir('%s') failed"), path); + return ret; +} + +static int em_rename_dir_from(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_FROM)); +} + +static int em_rename_dir_to(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVED_TO)); +} + +static int em_remove_watch(uint32_t mask) +{ + return (mask & IN_DELETE_SELF); +} + +static int em_dir_renamed(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_MOVE)); +} + +static int em_dir_created(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_CREATE)); +} + +static int em_dir_deleted(uint32_t mask) +{ + return ((mask & IN_ISDIR) && (mask & IN_DELETE)); +} + +static int em_force_shutdown(uint32_t mask) +{ + return (mask & IN_UNMOUNT) || (mask & IN_Q_OVERFLOW); +} + +static int em_ignore(uint32_t mask) +{ + return (mask & IN_IGNORED) || (mask & IN_MOVE_SELF); +} + +static void log_mask_set(const char *path, uint32_t mask) +{ + struct strbuf msg = STRBUF_INIT; + + if (mask & IN_ACCESS) + strbuf_addstr(&msg, "IN_ACCESS|"); + if (mask & IN_MODIFY) + strbuf_addstr(&msg, "IN_MODIFY|"); + if (mask & IN_ATTRIB) + strbuf_addstr(&msg, "IN_ATTRIB|"); + if (mask & IN_CLOSE_WRITE) + strbuf_addstr(&msg, "IN_CLOSE_WRITE|"); + if (mask & IN_CLOSE_NOWRITE) + strbuf_addstr(&msg, "IN_CLOSE_NOWRITE|"); + if (mask & IN_OPEN) + strbuf_addstr(&msg, "IN_OPEN|"); + if (mask & IN_MOVED_FROM) + strbuf_addstr(&msg, "IN_MOVED_FROM|"); + if (mask & IN_MOVED_TO) + strbuf_addstr(&msg, "IN_MOVED_TO|"); + if (mask & IN_CREATE) + strbuf_addstr(&msg, "IN_CREATE|"); + if (mask & IN_DELETE) + strbuf_addstr(&msg, "IN_DELETE|"); + if (mask & IN_DELETE_SELF) + strbuf_addstr(&msg, "IN_DELETE_SELF|"); + if (mask & IN_MOVE_SELF) + strbuf_addstr(&msg, "IN_MOVE_SELF|"); + if (mask & IN_UNMOUNT) + strbuf_addstr(&msg, "IN_UNMOUNT|"); + if (mask & IN_Q_OVERFLOW) + strbuf_addstr(&msg, "IN_Q_OVERFLOW|"); + if (mask & IN_IGNORED) + strbuf_addstr(&msg, "IN_IGNORED|"); + if (mask & IN_ISDIR) + strbuf_addstr(&msg, "IN_ISDIR|"); + + strbuf_strip_suffix(&msg, "|"); + + trace_printf_key(&trace_fsmonitor, "inotify_event: '%s', mask=%#8.8x %s", + path, mask, msg.buf); + + strbuf_release(&msg); +} + +int fsm_listen__ctor(struct fsmonitor_daemon_state *state) +{ + int fd; + int ret = 0; + struct fsm_listen_data *data; + + CALLOC_ARRAY(data, 1); + state->listen_data = data; + state->listen_error_code = -1; + data->fd_inotify = -1; + data->shutdown = SHUTDOWN_ERROR; + + fd = inotify_init1(O_NONBLOCK); + if (fd < 0) { + FREE_AND_NULL(state->listen_data); + return error_errno(_("inotify_init1() failed")); + } + + data->fd_inotify = fd; + + hashmap_init(&data->watches, watch_entry_cmp, NULL, 0); + hashmap_init(&data->renames, rename_entry_cmp, NULL, 0); + hashmap_init(&data->revwatches, revwatches_entry_cmp, NULL, 0); + + if (add_watch(state->path_worktree_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_worktree_watch.buf, state, NULL)) + ret = -1; + else if (state->nr_paths_watching > 1) { + if (add_watch(state->path_gitdir_watch.buf, data)) + ret = -1; + else if (register_inotify(state->path_gitdir_watch.buf, state, NULL)) + ret = -1; + } + + if (!ret) { + state->listen_error_code = 0; + data->shutdown = SHUTDOWN_CONTINUE; + } + + return ret; +} + +void fsm_listen__dtor(struct fsmonitor_daemon_state *state) +{ + struct fsm_listen_data *data; + struct hashmap_iter iter; + struct watch_entry *w; + struct watch_entry **to_remove; + size_t nr_to_remove = 0, alloc_to_remove = 0; + size_t i; + int fd; + + if (!state || !state->listen_data) + return; + + data = state->listen_data; + fd = data->fd_inotify; + + /* + * Collect all entries first, then remove them. + * We can't modify the hashmap while iterating over it. + */ + to_remove = NULL; + hashmap_for_each_entry(&data->watches, &iter, w, ent) { + ALLOC_GROW(to_remove, nr_to_remove + 1, alloc_to_remove); + to_remove[nr_to_remove++] = w; + } + + for (i = 0; i < nr_to_remove; i++) { + to_remove[i]->cookie = 0; /* ignore any pending renames */ + remove_watch(to_remove[i], data); + } + free(to_remove); + + hashmap_clear(&data->watches); + + hashmap_clear(&data->revwatches); /* remove_watch freed the entries */ + + hashmap_clear_and_free(&data->renames, struct rename_entry, ent); + + FREE_AND_NULL(state->listen_data); + + if (fd >= 0 && (close(fd) < 0)) + error_errno(_("closing inotify file descriptor failed")); +} + +void fsm_listen__stop_async(struct fsmonitor_daemon_state *state) +{ + if (state && state->listen_data && + state->listen_data->shutdown == SHUTDOWN_CONTINUE) + state->listen_data->shutdown = SHUTDOWN_STOP; +} + +/* + * Process a single inotify event and queue for publication. + */ +static int process_event(const char *path, + const struct inotify_event *event, + struct fsmonitor_batch **batch, + struct string_list *cookie_list, + struct fsmonitor_daemon_state *state) +{ + const char *rel; + const char *last_sep; + + switch (fsmonitor_classify_path_absolute(state, path)) { + case IS_INSIDE_DOT_GIT_WITH_COOKIE_PREFIX: + case IS_INSIDE_GITDIR_WITH_COOKIE_PREFIX: + /* Use just the filename of the cookie file. */ + last_sep = find_last_dir_sep(path); + string_list_append(cookie_list, + last_sep ? last_sep + 1 : path); + break; + case IS_INSIDE_DOT_GIT: + case IS_INSIDE_GITDIR: + break; + case IS_DOT_GIT: + case IS_GITDIR: + /* + * If .git directory is deleted or renamed away, + * we have to quit. + */ + if (em_dir_deleted(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir removed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + if (em_dir_renamed(event->mask)) { + trace_printf_key(&trace_fsmonitor, + "event: gitdir renamed"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + break; + case IS_WORKDIR_PATH: + /* normal events in the working directory */ + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set(path, event->mask); + + if (!*batch) + *batch = fsmonitor_batch__new(); + + rel = path + state->path_worktree_watch.len + 1; + fsmonitor_batch__add_path(*batch, rel); + + if (em_dir_deleted(event->mask)) + break; + + /* received IN_MOVE_FROM, add tracking for expected IN_MOVE_TO */ + if (em_rename_dir_from(event->mask)) + add_dir_rename(event->cookie, path, state->listen_data); + + /* received IN_MOVE_TO, update watch to reflect new path */ + if (em_rename_dir_to(event->mask)) { + rename_dir(event->cookie, path, state->listen_data); + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + + if (em_dir_created(event->mask)) { + if (add_watch(path, state->listen_data)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + if (register_inotify(path, state, *batch)) { + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + } + break; + case IS_OUTSIDE_CONE: + default: + trace_printf_key(&trace_fsmonitor, + "ignoring '%s'", path); + break; + } + return 0; +done: + return -1; +} + +/* + * Read the inotify event stream and pre-process events before further + * processing and eventual publishing. + */ +static void handle_events(struct fsmonitor_daemon_state *state) +{ + /* See https://man7.org/linux/man-pages/man7/inotify.7.html */ + char buf[4096] + __attribute__ ((aligned(__alignof__(struct inotify_event)))); + + struct hashmap *watches = &state->listen_data->watches; + struct fsmonitor_batch *batch = NULL; + struct string_list cookie_list = STRING_LIST_INIT_DUP; + struct watch_entry k, *w; + struct strbuf path = STRBUF_INIT; + const struct inotify_event *event; + int fd = state->listen_data->fd_inotify; + ssize_t len; + char *ptr, *p; + + for (;;) { + len = read(fd, buf, sizeof(buf)); + if (len == -1) { + if (errno == EAGAIN || errno == EINTR) + goto done; + error_errno(_("reading inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + goto done; + } + + /* nothing to read */ + if (len == 0) + goto done; + + /* Loop over all events in the buffer. */ + for (ptr = buf; ptr < buf + len; + ptr += sizeof(struct inotify_event) + event->len) { + + event = (const struct inotify_event *)ptr; + + if (em_ignore(event->mask)) + continue; + + /* File system was unmounted or event queue overflowed */ + if (em_force_shutdown(event->mask)) { + if (trace_pass_fl(&trace_fsmonitor)) + log_mask_set("forcing shutdown", event->mask); + state->listen_data->shutdown = SHUTDOWN_FORCE; + goto done; + } + + k.wd = event->wd; + hashmap_entry_init(&k.ent, memhash(&k.wd, sizeof(int))); + + w = hashmap_get_entry(watches, &k, ent, NULL); + if (!w) { + /* Watch was removed, skip event */ + continue; + } + + /* directory watch was removed */ + if (em_remove_watch(event->mask)) { + remove_watch(w, state->listen_data); + continue; + } + + strbuf_reset(&path); + strbuf_addf(&path, "%s/%s", w->dir, event->name); + + p = fsmonitor__resolve_alias(path.buf, &state->alias); + if (!p) + p = strbuf_detach(&path, NULL); + + if (process_event(p, event, &batch, &cookie_list, state)) { + free(p); + goto done; + } + free(p); + } + strbuf_reset(&path); + fsmonitor_publish(state, batch, &cookie_list); + string_list_clear(&cookie_list, 0); + batch = NULL; + } +done: + strbuf_release(&path); + fsmonitor_batch__free_list(batch); + string_list_clear(&cookie_list, 0); +} + +/* + * Non-blocking read of the inotify events stream. The inotify fd is polled + * frequently to help minimize the number of queue overflows. + */ +void fsm_listen__loop(struct fsmonitor_daemon_state *state) +{ + int poll_num; + /* + * Interval in seconds between checks for stale directory renames. + * A directory rename that is not completed within this window + * (i.e. no matching IN_MOVED_TO for an IN_MOVED_FROM) indicates + * missed events, forcing a shutdown. + */ + const int interval = 1; + time_t checked = time(NULL); + struct pollfd fds[1]; + + fds[0].fd = state->listen_data->fd_inotify; + fds[0].events = POLLIN; + + /* + * Our fs event listener is now running, so it's safe to start + * serving client requests. + */ + ipc_server_start_async(state->ipc_server_data); + + for (;;) { + switch (state->listen_data->shutdown) { + case SHUTDOWN_CONTINUE: + poll_num = poll(fds, 1, 50); + if (poll_num == -1) { + if (errno == EINTR) + continue; + error_errno(_("polling inotify message stream failed")); + state->listen_data->shutdown = SHUTDOWN_ERROR; + continue; + } + + if ((time(NULL) - checked) >= interval) { + checked = time(NULL); + if (check_stale_dir_renames(&state->listen_data->renames, + checked - interval)) { + trace_printf_key(&trace_fsmonitor, + "missed IN_MOVED_TO events, forcing shutdown"); + state->listen_data->shutdown = SHUTDOWN_FORCE; + continue; + } + } + + if (poll_num > 0 && (fds[0].revents & POLLIN)) + handle_events(state); + + continue; + case SHUTDOWN_ERROR: + state->listen_error_code = -1; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_FORCE: + state->listen_error_code = 0; + ipc_server_stop_async(state->ipc_server_data); + break; + case SHUTDOWN_STOP: + default: + state->listen_error_code = 0; + break; + } + return; + } +} diff --git a/compat/fsmonitor/fsm-path-utils-linux.c b/compat/fsmonitor/fsm-path-utils-linux.c new file mode 100644 index 0000000000..c9866b1b24 --- /dev/null +++ b/compat/fsmonitor/fsm-path-utils-linux.c @@ -0,0 +1,217 @@ +#include "git-compat-util.h" +#include "fsmonitor-ll.h" +#include "fsmonitor-path-utils.h" +#include "gettext.h" +#include "trace.h" + +#include <sys/statfs.h> + +#ifdef HAVE_LINUX_MAGIC_H +#include <linux/magic.h> +#endif + +/* + * Filesystem magic numbers for remote filesystems. + * Defined here if not available in linux/magic.h. + */ +#ifndef CIFS_SUPER_MAGIC +#define CIFS_SUPER_MAGIC 0xff534d42 +#endif +#ifndef SMB_SUPER_MAGIC +#define SMB_SUPER_MAGIC 0x517b +#endif +#ifndef SMB2_SUPER_MAGIC +#define SMB2_SUPER_MAGIC 0xfe534d42 +#endif +#ifndef NFS_SUPER_MAGIC +#define NFS_SUPER_MAGIC 0x6969 +#endif +#ifndef AFS_SUPER_MAGIC +#define AFS_SUPER_MAGIC 0x5346414f +#endif +#ifndef CODA_SUPER_MAGIC +#define CODA_SUPER_MAGIC 0x73757245 +#endif +#ifndef FUSE_SUPER_MAGIC +#define FUSE_SUPER_MAGIC 0x65735546 +#endif + +/* + * Check if filesystem type is a remote filesystem. + */ +static int is_remote_fs(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + case SMB_SUPER_MAGIC: + case SMB2_SUPER_MAGIC: + case NFS_SUPER_MAGIC: + case AFS_SUPER_MAGIC: + case CODA_SUPER_MAGIC: + case FUSE_SUPER_MAGIC: + return 1; + default: + return 0; + } +} + +/* + * Map filesystem magic numbers to human-readable names as a fallback + * when /proc/mounts is unavailable. This only covers the remote and + * special filesystems in is_remote_fs() above; local filesystems are + * never flagged as incompatible, so we do not need their names here. + */ +static const char *get_fs_typename(unsigned long f_type) +{ + switch (f_type) { + case CIFS_SUPER_MAGIC: + return "cifs"; + case SMB_SUPER_MAGIC: + return "smb"; + case SMB2_SUPER_MAGIC: + return "smb2"; + case NFS_SUPER_MAGIC: + return "nfs"; + case AFS_SUPER_MAGIC: + return "afs"; + case CODA_SUPER_MAGIC: + return "coda"; + case FUSE_SUPER_MAGIC: + return "fuse"; + default: + return "unknown"; + } +} + +/* + * Find the mount point for a given path by reading /proc/mounts. + * + * statfs(2) gives us f_type (the magic number) but not the human-readable + * filesystem type string. We scan /proc/mounts to find the mount entry + * whose path is the longest prefix of ours and whose f_fsid matches, + * which gives us the fstype string (e.g. "nfs", "ext4") for logging. + */ +static char *find_mount(const char *path, const struct statfs *path_fs) +{ + FILE *fp; + struct strbuf line = STRBUF_INIT; + struct strbuf match = STRBUF_INIT; + struct strbuf fstype = STRBUF_INIT; + char *result = NULL; + + fp = fopen("/proc/mounts", "r"); + if (!fp) + return NULL; + + while (strbuf_getline(&line, fp) != EOF) { + char *fields[6]; + char *p = line.buf; + int i; + + /* Parse mount entry: device mountpoint fstype options dump pass */ + for (i = 0; i < 6 && p; i++) { + fields[i] = p; + p = strchr(p, ' '); + if (p) + *p++ = '\0'; + } + + if (i >= 3) { + const char *mountpoint = fields[1]; + const char *type = fields[2]; + struct statfs mount_fs; + + /* Check if this mount point is a prefix of our path */ + if (starts_with(path, mountpoint) && + (path[strlen(mountpoint)] == '/' || + path[strlen(mountpoint)] == '\0')) { + /* Check if filesystem ID matches */ + if (statfs(mountpoint, &mount_fs) == 0 && + !memcmp(&mount_fs.f_fsid, &path_fs->f_fsid, + sizeof(mount_fs.f_fsid))) { + /* Keep the longest matching mount point */ + if (strlen(mountpoint) > match.len) { + strbuf_reset(&match); + strbuf_addstr(&match, mountpoint); + strbuf_reset(&fstype); + strbuf_addstr(&fstype, type); + } + } + } + } + } + + fclose(fp); + strbuf_release(&line); + strbuf_release(&match); + + if (fstype.len) + result = strbuf_detach(&fstype, NULL); + else + strbuf_release(&fstype); + + return result; +} + +int fsmonitor__get_fs_info(const char *path, struct fs_info *fs_info) +{ + struct statfs fs; + + if (statfs(path, &fs) == -1) { + int saved_errno = errno; + trace_printf_key(&trace_fsmonitor, "statfs('%s') failed: %s", + path, strerror(saved_errno)); + errno = saved_errno; + return -1; + } + + trace_printf_key(&trace_fsmonitor, + "statfs('%s') [type 0x%08lx]", + path, (unsigned long)fs.f_type); + + fs_info->is_remote = is_remote_fs(fs.f_type); + + /* + * Try to get filesystem type from /proc/mounts for a more + * descriptive name. + */ + fs_info->typename = find_mount(path, &fs); + if (!fs_info->typename) + fs_info->typename = xstrdup(get_fs_typename(fs.f_type)); + + trace_printf_key(&trace_fsmonitor, + "'%s' is_remote: %d, typename: %s", + path, fs_info->is_remote, fs_info->typename); + + return 0; +} + +int fsmonitor__is_fs_remote(const char *path) +{ + struct fs_info fs; + + if (fsmonitor__get_fs_info(path, &fs)) + return -1; + + free(fs.typename); + + return fs.is_remote; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +int fsmonitor__get_alias(const char *path UNUSED, + struct alias_info *info UNUSED) +{ + return 0; +} + +/* + * No-op for Linux - we don't have firmlinks like macOS. + */ +char *fsmonitor__resolve_alias(const char *path UNUSED, + const struct alias_info *info UNUSED) +{ + return NULL; +} diff --git a/config.mak.uname b/config.mak.uname index 33877020e9..638f7e1bde 100644 --- a/config.mak.uname +++ b/config.mak.uname @@ -68,6 +68,16 @@ ifeq ($(uname_S),Linux) BASIC_CFLAGS += -std=c99 endif LINK_FUZZ_PROGRAMS = YesPlease + + # The builtin FSMonitor on Linux builds upon Simple-IPC. Both require + # Unix domain sockets and PThreads. + ifndef NO_PTHREADS + ifndef NO_UNIX_SOCKETS + FSMONITOR_DAEMON_BACKEND = linux + FSMONITOR_OS_SETTINGS = unix + BASIC_CFLAGS += -DHAVE_LINUX_MAGIC_H + endif + endif endif ifeq ($(uname_S),GNU/kFreeBSD) HAVE_ALLOCA_H = YesPlease diff --git a/contrib/buildsystems/CMakeLists.txt b/contrib/buildsystems/CMakeLists.txt index d613809e26..b7da108f29 100644 --- a/contrib/buildsystems/CMakeLists.txt +++ b/contrib/buildsystems/CMakeLists.txt @@ -296,6 +296,10 @@ if(SUPPORTS_SIMPLE_IPC) elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin") set(FSMONITOR_DAEMON_BACKEND "darwin") set(FSMONITOR_OS_SETTINGS "unix") + elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") + set(FSMONITOR_DAEMON_BACKEND "linux") + set(FSMONITOR_OS_SETTINGS "unix") + add_compile_definitions(HAVE_LINUX_MAGIC_H) endif() if(FSMONITOR_DAEMON_BACKEND) @@ -1149,8 +1153,8 @@ endif() file(STRINGS ${CMAKE_SOURCE_DIR}/GIT-BUILD-OPTIONS.in git_build_options NEWLINE_CONSUME) string(REPLACE "@BROKEN_PATH_FIX@" "" git_build_options "${git_build_options}") string(REPLACE "@DIFF@" "'${DIFF}'" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "win32" git_build_options "${git_build_options}") -string(REPLACE "@FSMONITOR_OS_SETTINGS@" "win32" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_DAEMON_BACKEND@" "${FSMONITOR_DAEMON_BACKEND}" git_build_options "${git_build_options}") +string(REPLACE "@FSMONITOR_OS_SETTINGS@" "${FSMONITOR_OS_SETTINGS}" git_build_options "${git_build_options}") string(REPLACE "@GITWEBDIR@" "'${GITWEBDIR}'" git_build_options "${git_build_options}") string(REPLACE "@GIT_INTEROP_MAKE_OPTS@" "" git_build_options "${git_build_options}") string(REPLACE "@GIT_PERF_LARGE_REPO@" "" git_build_options "${git_build_options}") diff --git a/meson.build b/meson.build index 4f0c0a33b8..123d218460 100644 --- a/meson.build +++ b/meson.build @@ -1324,6 +1324,10 @@ fsmonitor_os = '' if host_machine.system() == 'windows' fsmonitor_backend = 'win32' fsmonitor_os = 'win32' +elif host_machine.system() == 'linux' and threads.found() and compiler.has_header('linux/magic.h') + fsmonitor_backend = 'linux' + fsmonitor_os = 'unix' + libgit_c_args += '-DHAVE_LINUX_MAGIC_H' elif host_machine.system() == 'darwin' fsmonitor_backend = 'darwin' fsmonitor_os = 'unix' -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 08/12] run-command: add close_fd_above_stderr option 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (6 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget ` (4 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a close_fd_above_stderr flag to struct child_process. When set, the child closes file descriptors 3 and above between fork and exec (skipping the child-notifier pipe), capped at sysconf(_SC_OPEN_MAX) or 4096, whichever is smaller. This prevents the child from inheriting pipe endpoints or other descriptors from the parent environment (e.g., the test harness). Signed-off-by: Paul Tarjan <github@paulisageek.com> --- run-command.c | 12 ++++++++++++ run-command.h | 9 +++++++++ 2 files changed, 21 insertions(+) diff --git a/run-command.c b/run-command.c index e3e02475cc..f4361906c9 100644 --- a/run-command.c +++ b/run-command.c @@ -546,6 +546,7 @@ static void atfork_parent(struct atfork_state *as) "restoring signal mask"); #endif } + #endif /* GIT_WINDOWS_NATIVE */ static inline void set_cloexec(int fd) @@ -832,6 +833,17 @@ fail_pipe: child_close(cmd->out); } + if (cmd->close_fd_above_stderr) { + long max_fd = sysconf(_SC_OPEN_MAX); + int fd; + if (max_fd < 0 || max_fd > 4096) + max_fd = 4096; + for (fd = 3; fd < max_fd; fd++) { + if (fd != child_notifier) + close(fd); + } + } + if (cmd->dir && chdir(cmd->dir)) child_die(CHILD_ERR_CHDIR); diff --git a/run-command.h b/run-command.h index 0df25e445f..fdaa01e140 100644 --- a/run-command.h +++ b/run-command.h @@ -141,6 +141,15 @@ struct child_process { unsigned stdout_to_stderr:1; unsigned clean_on_exit:1; unsigned wait_after_clean:1; + + /** + * Close file descriptors 3 and above in the child after forking + * but before exec. This prevents the child from inheriting + * pipe endpoints or other descriptors from the parent + * environment (e.g., the test harness). + */ + unsigned close_fd_above_stderr:1; + void (*clean_on_exit_handler)(struct child_process *process); }; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 09/12] fsmonitor: close inherited file descriptors and detach in daemon 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (7 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 08/12] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget ` (3 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> When the fsmonitor daemon is spawned as a background process, it may inherit file descriptors from its parent that it does not need. In particular, when the test harness or a CI system captures output through pipes, the daemon can inherit duplicated pipe endpoints. If the daemon holds these open, the parent process never sees EOF and may appear to hang. Set close_fd_above_stderr on the child process at both daemon startup paths: the explicit "fsmonitor--daemon start" command and the implicit spawn triggered by fsmonitor-ipc when a client finds no running daemon. Also suppress stdout and stderr on the implicit spawn path to prevent the background daemon from writing to the client's terminal. Additionally, call setsid() when the daemon starts with --detach to create a new session and process group. This prevents the daemon from being part of the spawning shell's process group, which could cause the shell's "wait" to block until the daemon exits. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 16 ++++++++++++++-- fsmonitor-ipc.c | 3 +++ 2 files changed, 17 insertions(+), 2 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index c8ec7b722e..b2a816dc3f 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -1439,7 +1439,7 @@ done: return err; } -static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) +static int try_to_run_foreground_daemon(int detach_console) { /* * Technically, we don't need to probe for an existing daemon @@ -1459,10 +1459,21 @@ static int try_to_run_foreground_daemon(int detach_console MAYBE_UNUSED) fflush(stderr); } + if (detach_console) { #ifdef GIT_WINDOWS_NATIVE - if (detach_console) FreeConsole(); +#else + /* + * Create a new session so that the daemon is detached + * from the parent's process group. This prevents + * shells with job control (e.g. bash with "set -m") + * from waiting on the daemon when they wait for a + * foreground command that implicitly spawned it. + */ + if (setsid() == -1) + warning_errno(_("setsid failed")); #endif + } return !!fsmonitor_run_daemon(); } @@ -1525,6 +1536,7 @@ static int try_to_start_background_daemon(void) cp.no_stdin = 1; cp.no_stdout = 1; cp.no_stderr = 1; + cp.close_fd_above_stderr = 1; sbgr = start_bg_command(&cp, bg_wait_cb, NULL, fsmonitor__start_timeout_sec); diff --git a/fsmonitor-ipc.c b/fsmonitor-ipc.c index f1b1631111..6112d13064 100644 --- a/fsmonitor-ipc.c +++ b/fsmonitor-ipc.c @@ -61,6 +61,9 @@ static int spawn_daemon(void) cmd.git_cmd = 1; cmd.no_stdin = 1; + cmd.no_stdout = 1; + cmd.no_stderr = 1; + cmd.close_fd_above_stderr = 1; cmd.trace2_child_class = "fsmonitor"; strvec_pushl(&cmd.args, "fsmonitor--daemon", "start", NULL); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 10/12] fsmonitor: add timeout to daemon stop command 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (8 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget ` (2 subsequent siblings) 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> The "fsmonitor--daemon stop" command polls in a loop waiting for the daemon to exit after sending a "quit" command over IPC. If the daemon fails to shut down (e.g. it is stuck or wedged), this loop spins forever. Add a 30-second timeout so the stop command returns an error instead of blocking indefinitely. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index b2a816dc3f..53d8ad1f0d 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -86,6 +86,8 @@ static int do_as_client__send_stop(void) { struct strbuf answer = STRBUF_INIT; int ret; + int max_wait_ms = 30000; + int elapsed_ms = 0; ret = fsmonitor_ipc__send_command("quit", &answer); @@ -96,8 +98,16 @@ static int do_as_client__send_stop(void) return ret; trace2_region_enter("fsm_client", "polling-for-daemon-exit", NULL); - while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) + while (fsmonitor_ipc__get_state() == IPC_STATE__LISTENING) { + if (elapsed_ms >= max_wait_ms) { + trace2_region_leave("fsm_client", + "polling-for-daemon-exit", NULL); + return error(_("daemon did not stop within %d seconds"), + max_wait_ms / 1000); + } sleep_millisec(50); + elapsed_ms += 50; + } trace2_region_leave("fsm_client", "polling-for-daemon-exit", NULL); return 0; -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 11/12] fsmonitor: add tests for Linux 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (9 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 7:37 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Patrick Steinhardt 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Add a smoke test that verifies the filesystem actually delivers inotify events to the daemon. On some configurations (e.g., overlayfs with older kernels), inotify watches succeed but events are never delivered. The daemon cookie wait will time out, but every subsequent test would fail. Skip the entire test file early when this is detected. Add a test that exercises rapid nested directory creation to verify the daemon correctly handles the EEXIST race between recursive scan and queued inotify events. When IN_MASK_CREATE is available and a directory watch is added during recursive registration, the kernel may also deliver a queued IN_CREATE event for the same directory. The second inotify_add_watch() returns EEXIST, which must be treated as harmless. An earlier version of the listener crashed in this scenario. Reduce --start-timeout from the default 60 seconds to 10 seconds so that tests fail promptly when the daemon cannot start. Harden the test helpers to work in environments without procps (e.g., Fedora CI): fall back to reading /proc/$pid/stat for the process group ID when ps is unavailable, guard stop_git() against an empty pgid, and redirect stderr from kill to /dev/null to avoid noise when processes have already exited. Use set -m to enable job control in the submodule-pull test so that the background git pull gets its own process group, preventing the shell wait from blocking on the daemon. setsid() in the previous commit detaches the daemon itself, but the intermediate git pull process still needs its own process group for the test shell to manage it correctly. Signed-off-by: Paul Tarjan <github@paulisageek.com> --- t/t7527-builtin-fsmonitor.sh | 89 +++++++++++++++++++++++++++++++++--- 1 file changed, 82 insertions(+), 7 deletions(-) diff --git a/t/t7527-builtin-fsmonitor.sh b/t/t7527-builtin-fsmonitor.sh index 409cd0cd12..774da5ac60 100755 --- a/t/t7527-builtin-fsmonitor.sh +++ b/t/t7527-builtin-fsmonitor.sh @@ -10,9 +10,58 @@ then test_done fi +# Verify that the filesystem delivers events to the daemon. +# On some configurations (e.g., overlayfs with older kernels), +# inotify watches succeed but events are never delivered. The +# cookie wait will time out and the daemon logs a trace message. +# +# Use "timeout" (if available) to guard each step against hangs. +maybe_timeout () { + if type timeout >/dev/null 2>&1 + then + timeout "$@" + else + shift + "$@" + fi +} +verify_fsmonitor_works () { + git init test_fsmonitor_smoke || return 1 + + GIT_TRACE_FSMONITOR="$PWD/smoke.trace" && + export GIT_TRACE_FSMONITOR && + maybe_timeout 30 \ + git -C test_fsmonitor_smoke fsmonitor--daemon start \ + --start-timeout=10 + ret=$? + unset GIT_TRACE_FSMONITOR + if test $ret -ne 0 + then + rm -rf test_fsmonitor_smoke smoke.trace + return 1 + fi + + maybe_timeout 10 \ + test-tool -C test_fsmonitor_smoke fsmonitor-client query \ + --token 0 >/dev/null 2>&1 + maybe_timeout 5 \ + git -C test_fsmonitor_smoke fsmonitor--daemon stop 2>/dev/null + ! grep -q "cookie_wait timed out" "$PWD/smoke.trace" 2>/dev/null + ret=$? + rm -rf test_fsmonitor_smoke smoke.trace + return $ret +} + +if ! verify_fsmonitor_works +then + skip_all="filesystem does not deliver fsmonitor events (container/overlayfs?)" + test_done +fi + stop_daemon_delete_repo () { r=$1 && - test_might_fail git -C $r fsmonitor--daemon stop && + test_might_fail maybe_timeout 30 \ + git -C $r fsmonitor--daemon stop 2>/dev/null rm -rf $1 } @@ -67,7 +116,7 @@ start_daemon () { export GIT_TEST_FSMONITOR_TOKEN fi && - git $r fsmonitor--daemon start && + git $r fsmonitor--daemon start --start-timeout=10 && git $r fsmonitor--daemon status ) } @@ -520,6 +569,28 @@ test_expect_success 'directory changes to a file' ' grep "^event: dir1$" .git/trace ' +test_expect_success 'rapid nested directory creation' ' + test_when_finished "git fsmonitor--daemon stop; rm -rf rapid" && + + start_daemon --tf "$PWD/.git/trace" && + + # Rapidly create nested directories to exercise race conditions + # where directory watches may be added concurrently during + # event processing and recursive scanning. + for i in $(test_seq 1 20) + do + mkdir -p "rapid/nested/dir$i/subdir/deep" || return 1 + done && + + # Give the daemon time to process all events + sleep 1 && + + test-tool fsmonitor-client query --token 0 && + + # Verify daemon is still running (did not crash) + git fsmonitor--daemon status +' + # The next few test cases exercise the token-resync code. When filesystem # drops events (because of filesystem velocity or because the daemon isn't # polling fast enough), we need to discard the cached data (relative to the @@ -910,7 +981,10 @@ test_expect_success "submodule absorbgitdirs implicitly starts daemon" ' start_git_in_background () { git "$@" & git_pid=$! - git_pgid=$(ps -o pgid= -p $git_pid) + git_pgid=$(ps -o pgid= -p $git_pid 2>/dev/null || + awk '{print $5}' /proc/$git_pid/stat 2>/dev/null) && + git_pgid="${git_pgid## }" && + git_pgid="${git_pgid%% }" nr_tries_left=10 while true do @@ -921,15 +995,16 @@ start_git_in_background () { fi sleep 1 nr_tries_left=$(($nr_tries_left - 1)) - done >/dev/null 2>&1 & + done >/dev/null 2>&1 3>&- 4>&- 5>&- 6>&- 7>&- & watchdog_pid=$! wait $git_pid } stop_git () { - while kill -0 -- -$git_pgid + test -n "$git_pgid" || return 0 + while kill -0 -- -$git_pgid 2>/dev/null do - kill -- -$git_pgid + kill -- -$git_pgid 2>/dev/null sleep 1 done } @@ -944,7 +1019,7 @@ stop_watchdog () { test_expect_success !MINGW "submodule implicitly starts daemon by pull" ' test_atexit "stop_watchdog" && - test_when_finished "stop_git; rm -rf cloned super sub" && + test_when_finished "set +m; stop_git; rm -rf cloned super sub" && create_super super && create_sub sub && -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* [PATCH v11 12/12] fsmonitor: convert shown khash to strset in do_handle_client 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (10 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 ` Paul Tarjan via GitGitGadget 2026-03-05 7:37 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Patrick Steinhardt 12 siblings, 0 replies; 129+ messages in thread From: Paul Tarjan via GitGitGadget @ 2026-03-05 6:55 UTC (permalink / raw) To: git; +Cc: Patrick Steinhardt, Paul Tarjan, Paul Tarjan, Paul Tarjan From: Paul Tarjan <github@paulisageek.com> Replace the khash-based string set used for deduplicating pathnames in do_handle_client() with a strset, which provides a cleaner interface for the same purpose. Since the paths are interned strings from the batch data, use strdup_strings=0 to avoid unnecessary copies. Suggested-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Paul Tarjan <github@paulisageek.com> --- builtin/fsmonitor--daemon.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/builtin/fsmonitor--daemon.c b/builtin/fsmonitor--daemon.c index 53d8ad1f0d..f920cf3a82 100644 --- a/builtin/fsmonitor--daemon.c +++ b/builtin/fsmonitor--daemon.c @@ -16,7 +16,7 @@ #include "fsmonitor--daemon.h" #include "simple-ipc.h" -#include "khash.h" +#include "strmap.h" #include "run-command.h" #include "trace.h" #include "trace2.h" @@ -674,8 +674,6 @@ static int fsmonitor_parse_client_token(const char *buf_token, return 0; } -KHASH_INIT(str, const char *, int, 0, kh_str_hash_func, kh_str_hash_equal) - static int do_handle_client(struct fsmonitor_daemon_state *state, const char *command, ipc_server_reply_cb *reply, @@ -692,8 +690,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const struct fsmonitor_batch *batch; struct fsmonitor_batch *remainder = NULL; intmax_t count = 0, duplicates = 0; - kh_str_t *shown = NULL; - int hash_ret; + struct strset shown = STRSET_INIT; int do_trivial = 0; int do_flush = 0; int do_cookie = 0; @@ -882,14 +879,14 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, * so walk the batch list backwards from the current head back * to the batch (sequence number) they named. * - * We use khash to de-dup the list of pathnames. + * We use a strset to de-dup the list of pathnames. * * NEEDSWORK: each batch contains a list of interned strings, * so we only need to do pointer comparisons here to build the * hash table. Currently, we're still comparing the string * values. */ - shown = kh_init_str(); + strset_init_with_options(&shown, NULL, 0); for (batch = batch_head; batch && batch->batch_seq_nr > requested_oldest_seq_nr; batch = batch->next) { @@ -899,11 +896,9 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, const char *s = batch->interned_paths[k]; size_t s_len; - if (kh_get_str(shown, s) != kh_end(shown)) + if (!strset_add(&shown, s)) duplicates++; else { - kh_put_str(shown, s, &hash_ret); - trace_printf_key(&trace_fsmonitor, "send[%"PRIuMAX"]: %s", count, s); @@ -973,7 +968,7 @@ static int do_handle_client(struct fsmonitor_daemon_state *state, trace2_data_intmax("fsmonitor", the_repository, "response/count/duplicates", duplicates); cleanup: - kh_destroy_str(shown); + strset_clear(&shown); strbuf_release(&response_token); strbuf_release(&requested_token_id); strbuf_release(&payload); -- gitgitgadget ^ permalink raw reply related [flat|nested] 129+ messages in thread
* Re: [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget ` (11 preceding siblings ...) 2026-03-05 6:55 ` [PATCH v11 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget @ 2026-03-05 7:37 ` Patrick Steinhardt 2026-03-05 14:15 ` Paul Tarjan 2026-03-25 20:00 ` Junio C Hamano 12 siblings, 2 replies; 129+ messages in thread From: Patrick Steinhardt @ 2026-03-05 7:37 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan, Paul Tarjan On Thu, Mar 05, 2026 at 06:55:00AM +0000, Paul Tarjan via GitGitGadget wrote: > Changes since v10: > > * Reverted pre_exec_cb callback back to simple close_fd_above_stderr flag > per Junio's clarification (same as v8) One ask from my side: I would welcome it if you slowed down a bit with sending out new versions. Sending three different versions within a couple hours without any reviews in between makes me pause, as I have no idea whether the current version is good to be reviewed or whether I should expect another 5 rerolls. So please take a bit more time to work feedback into your patch series before sending out the next version. It's totally fine to wait a couple days between iterations. Also, could you please clarify whether the patch series has been written by AI and if so, which parts of it are? Thanks! Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 7:37 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Patrick Steinhardt @ 2026-03-05 14:15 ` Paul Tarjan 2026-03-25 20:00 ` Junio C Hamano 1 sibling, 0 replies; 129+ messages in thread From: Paul Tarjan @ 2026-03-05 14:15 UTC (permalink / raw) To: ps; +Cc: git, gitster, paul, Paul Tarjan Patrick Steinhardt <ps@pks.im> writes: > One ask from my side: I would welcome it if you slowed down a bit with > sending out new versions. Sending three different versions within a > couple hours without any reviews in between makes me pause, as I have no > idea whether the current version is good to be reviewed or whether I > should expect another 5 rerolls. Apologies for that. I thought integrating the feedback would be a quick fix since it was small and worked locally so I /submitted it without waiting for a Windows build. When it broke I fixed the Windows version, saw it passed and typed /submit. Then I got the feedback on the mailing list that it wasn't what Junio wanted so I undid that and noted in my front matter that (v11 == v8). > So please take a bit more time to work feedback into your patch series > before sending out the next version. It's totally fine to wait a couple > days between iterations. Sorry for the quick submissions. I'm more used to the Github development flow where quick iteration on feedback is optimal to get a PR merged. > Also, could you please clarify whether the patch series has been written > by AI and if so, which parts of it are? I'm an Engineer at Anthropic so I've sadly completely lost my vim muscle that I built up over 20 years. All the file fwrite() operations were executed by my Claude wrapper but fully directed and iterated by me, with every line reviewed by me. That's just how I code nowadays. Claude is also an excellent code reviewer so it found multiple bugs in my code too. I'm aware of your policy and think I'm complying with it, but please let me know if you prefer something else: https://git-scm.com/docs/SubmittingPatches#ai These email replies are written by me but sent by Claude calling git commands as I've messed up formatting in the past trying to use Gmail and couldn't figure out the right git incantation to send to the mailing list, but Claude could. > Thanks! > > Patrick ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux 2026-03-05 7:37 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Patrick Steinhardt 2026-03-05 14:15 ` Paul Tarjan @ 2026-03-25 20:00 ` Junio C Hamano 1 sibling, 0 replies; 129+ messages in thread From: Junio C Hamano @ 2026-03-25 20:00 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget, Patrick Steinhardt Cc: git, Paul Tarjan, Paul Tarjan Patrick Steinhardt <ps@pks.im> writes: > On Thu, Mar 05, 2026 at 06:55:00AM +0000, Paul Tarjan via GitGitGadget wrote: >> Changes since v10: >> >> * Reverted pre_exec_cb callback back to simple close_fd_above_stderr flag >> per Junio's clarification (same as v8) > > One ask from my side: I would welcome it if you slowed down a bit with > sending out new versions. Sending three different versions within a > couple hours without any reviews in between makes me pause, as I have no > idea whether the current version is good to be reviewed or whether I > should expect another 5 rerolls. > > So please take a bit more time to work feedback into your patch series > before sending out the next version. It's totally fine to wait a couple > days between iterations. > > Also, could you please clarify whether the patch series has been written > by AI and if so, which parts of it are? > > Thanks! > > Patrick The topic went totally silent after this message; is this topic still viable? Thanks. ^ permalink raw reply [flat|nested] 129+ messages in thread
* Re: [PATCH v2] fsmonitor: implement filesystem change listener for Linux 2025-12-30 12:08 ` [PATCH v2] " Paul Tarjan via GitGitGadget 2025-12-30 12:55 ` [PATCH v3] " Paul Tarjan via GitGitGadget @ 2025-12-30 15:37 ` Junio C Hamano 1 sibling, 0 replies; 129+ messages in thread From: Junio C Hamano @ 2025-12-30 15:37 UTC (permalink / raw) To: Paul Tarjan via GitGitGadget; +Cc: git, Paul Tarjan Thanks for a quick turnaround, but it would be more efficient if you hunted all the leaks yourself, instead of getting a report for one issue and updating the patch to fix that one issue. Here is what I am getting these: $ make SANITIZE=address CC=clang && cd t && sh t7527-*.sh -i -v Note that "-i" is to say "stop at the first one". expecting success of 7527.12 'create some files': test_when_finished clean_up_repo_and_stop_daemon && start_daemon --tf "$PWD/.git/trace" && create_files && test-tool fsmonitor-client query --token 0 && grep "^event: dir1/new$" .git/trace && grep "^event: dir2/new$" .git/trace && grep "^event: new$" .git/trace fsmonitor-daemon is watching '/home/gitster/w/git.git/t/trash directory.t7527-builtin-fsmonitor' builtin:0.1039108.20251230T123036.129805Z:0/event: dir1/new event: dir1/new event: dir2/new event: dir2/new event: new event: new HEAD is now at 1d1edcb initial Removing dir1/new Removing dir2/new Removing new not ok 12 - create some files # # test_when_finished clean_up_repo_and_stop_daemon && # # start_daemon --tf "$PWD/.git/trace" && # # create_files && # # test-tool fsmonitor-client query --token 0 && # # grep "^event: dir1/new$" .git/trace && # grep "^event: dir2/new$" .git/trace && # grep "^event: new$" .git/trace # 1..12 ================================================================= ==git==1039073==ERROR: LeakSanitizer: detected memory leaks Direct leak of 40 byte(s) in 1 object(s) allocated from: #0 0x55c18d8d4042 in calloc (git+0x8c042) (BuildId: 4097db008a82663ae0b3398128a7ab4e09bbdd21) #1 0x55c18dc10f14 in xcalloc wrapper.c:154:8 #2 0x55c18d945f72 in kh_init_str builtin/fsmonitor--daemon.c:656:1 #3 0x55c18d945828 in do_handle_client builtin/fsmonitor--daemon.c:871:10 #4 0x55c18d945191 in handle_client builtin/fsmonitor--daemon.c:987:11 #5 0x55c18dc283e2 in worker_thread__do_io compat/simple-ipc/ipc-unix-socket.c:532:9 #6 0x55c18dc27a7f in worker_thread_proc compat/simple-ipc/ipc-unix-socket.c:606:9 #7 0x55c18d8d64f4 in void* ThreadStartFunc<false>(void*) lsan_interceptors.cpp.o #8 0x7fe358257b7a in start_thread nptl/pthread_create.c:448:8 #9 0x7fe3582d57b7 in __GI___clone3 misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:78 ^ permalink raw reply [flat|nested] 129+ messages in thread
end of thread, other threads:[~2026-03-25 20:00 UTC | newest] Thread overview: 129+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2025-12-30 8:14 [PATCH] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2025-12-30 11:38 ` Junio C Hamano 2025-12-30 12:08 ` [PATCH v2] " Paul Tarjan via GitGitGadget 2025-12-30 12:55 ` [PATCH v3] " Paul Tarjan via GitGitGadget 2025-12-31 17:41 ` [PATCH v4] " Paul Tarjan via GitGitGadget 2026-01-05 12:07 ` Patrick Steinhardt 2026-02-20 22:18 ` Junio C Hamano 2026-02-21 16:15 ` Paul Tarjan 2026-02-21 17:07 ` Junio C Hamano 2026-02-23 6:34 ` Patrick Steinhardt 2026-02-23 15:42 ` Junio C Hamano 2026-02-23 15:46 ` Patrick Steinhardt 2026-02-24 1:34 ` Paul Tarjan 2026-02-24 8:03 ` Patrick Steinhardt 2026-02-24 1:31 ` [PATCH v5] " Paul Tarjan via GitGitGadget 2026-02-24 8:03 ` Patrick Steinhardt 2026-02-25 20:17 ` [PATCH v6 00/10] " Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-02-25 21:01 ` Junio C Hamano 2026-02-25 20:17 ` [PATCH v6 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 03/10] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget 2026-02-25 21:13 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 2026-02-27 16:44 ` Junio C Hamano 2026-02-28 0:28 ` Paul Tarjan 2026-02-25 21:17 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 2026-02-25 20:17 ` [PATCH v6 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget 2026-02-25 21:30 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 2026-02-25 20:17 ` [PATCH v6 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget 2026-02-25 21:31 ` Junio C Hamano 2026-02-27 6:31 ` Paul Tarjan 2026-02-25 20:17 ` [PATCH v6 07/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 08/10] fsmonitor: add tests " Paul Tarjan via GitGitGadget 2026-02-25 20:17 ` [PATCH v6 09/10] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget 2026-02-25 21:41 ` Junio C Hamano 2026-02-25 20:17 ` [PATCH v6 10/10] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 01/10] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 02/10] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 03/10] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 04/10] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 05/10] fsmonitor: deduplicate IPC path logic for Unix platforms Paul Tarjan via GitGitGadget 2026-03-04 7:42 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 06/10] fsmonitor: deduplicate settings " Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 07/10] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 08/10] fsmonitor: add tests " Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 0:27 ` [PATCH v7 09/10] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget 2026-02-26 0:27 ` [PATCH v7 10/10] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget 2026-03-04 7:43 ` Patrick Steinhardt 2026-03-04 18:17 ` Paul Tarjan 2026-02-26 15:34 ` [PATCH v7 00/10] fsmonitor: implement filesystem change listener for Linux Junio C Hamano 2026-03-04 18:15 ` [PATCH v8 00/12] " Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 08/12] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget 2026-03-04 20:51 ` Junio C Hamano 2026-03-05 0:49 ` [PATCH v8 09/12] " Paul Tarjan 2026-03-05 4:13 ` Junio C Hamano 2026-03-05 6:38 ` [PATCH v9 09/12] run-command: add pre-exec callback for child processes Paul Tarjan 2026-03-04 18:15 ` [PATCH v8 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget 2026-03-04 18:15 ` [PATCH v8 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 08/12] run-command: add pre-exec callback for child processes Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget 2026-03-05 0:51 ` [PATCH v9 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget 2026-03-05 0:52 ` [PATCH v9 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 08/12] run-command: add pre-exec callback for child processes Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget 2026-03-05 1:16 ` [PATCH v10 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 01/12] fsmonitor: fix khash memory leak in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 02/12] fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 03/12] compat/win32: add pthread_cond_timedwait Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 04/12] fsmonitor: use pthread_cond_timedwait for cookie wait Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 05/12] fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 06/12] fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 07/12] fsmonitor: implement filesystem change listener for Linux Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 08/12] run-command: add close_fd_above_stderr option Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 09/12] fsmonitor: close inherited file descriptors and detach in daemon Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 10/12] fsmonitor: add timeout to daemon stop command Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 11/12] fsmonitor: add tests for Linux Paul Tarjan via GitGitGadget 2026-03-05 6:55 ` [PATCH v11 12/12] fsmonitor: convert shown khash to strset in do_handle_client Paul Tarjan via GitGitGadget 2026-03-05 7:37 ` [PATCH v11 00/12] fsmonitor: implement filesystem change listener for Linux Patrick Steinhardt 2026-03-05 14:15 ` Paul Tarjan 2026-03-25 20:00 ` Junio C Hamano 2025-12-30 15:37 ` [PATCH v2] " Junio C Hamano
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox