git commit hangs when run concurrently on macos

git commit hangs when run concurrently on macos

[Bennet Huber]

Thank you for filling out a Git bug report!
Please answer the following questions to help us understand your issue.

What did you do before the bug happened? (Steps to reproduce your issue)

Ran several "git commit" commands in parallel on separate repos.
Here's a simple test that does that.

```bash
dir-foreach-parallel () {
    subdirs=("${(@f)$(find ./ -maxdepth 1 -mindepth 1 -type d)}")
    for d in $subdirs
    do
        echo "$d"
        pushd "$d" >/dev/null
        $@ &
        popd >/dev/null
    done
}

mkdir -p /tmp/gitTest
cd /tmp/gitTest
for ((i=0;i<100;i++)) do mkdir git_$i; done

runTest() {
    git init
    USE_FS_MONITOR=true
    echo -e "[user]\n\tname = Gopher\n\temail =
gopher@example.com\n[core]\n\tfsmonitor = $USE_FS_MONITOR\n"
>.git/config
    GIT_TRACE=1 git commit --allow-empty -m `pwd` &>gitci.log
}

dir-foreach-parallel runTest
```

After running this, wait for output to subside, then run `jobs`.
On my machine, I usually see 5-10 hung "git commit" jobs.
Running `ps -efww | grep "git commit"` will show the repos they belong
to, which you can then use to
examine the corresponding gitci.log files.  The last line of all the
files I examined for hung "git
commit" commands is:

> 3:39:04.020533 run-command.c:758       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git fsmonitor--daemon run --detach --ipc-threads=8

Setting `USE_FS_MONITOR=false` in the code above consistently results
in no hung "git commit" commands, which leads me to suspect the
problem has to do with the fsmonitor feature.  It's also a decent,
though difficult to discover, workaround.


What did you expect to happen? (Expected behavior)
Concurrent git commands in separate repos shouldn't interfere with each other.

What happened instead? (Actual behavior)
Several "git commit" commands hung and failed to complete due to
fsmonitor not starting.

What's different between what you expected and what actually happened?
Success

Anything else you want to add:
* I tried to repro this on a linux box and failed. It seems to have
something to do with the macos
fsmonitor implementation.
  * `uname -a` of linux box that succeeds: Linux <hostname>
6.8.0-1016-aws #17-Ubuntu SMP Mon Sep  2 13:48:07 UTC 2024 x86_64
x86_64 x86_64 GNU/Linux
* I'm running an ARM Apple M3 Pro chip, so maybe ARM specific?
* The only mac I have access to is a corporate one, which has a ton of
security monitoring software running on it in the background.  It's
conceivable this could be interfering with fsmonitor somehow.  I
unfortunately can't test this on a clean mac.
* This use case may seem contrived, but I actually lost a lot of time
because of it.  I was trying
  to run some unit tests in parallel which involved running "git
commit" commands.  It took me
a long time to figure out why they were randomly hanging.  In
addition, I've previously used the `dir-foreach-parallel git
commit...` pattern for actual development, when working on software
which requires changes across several dependent git repos.

Please review the rest of the bug report below.
You can delete any lines you don't wish to share.


[System Info]
git version:
git version 2.46.2
cpu: arm64
no commit associated with this build
sizeof-long: 8
sizeof-size_t: 8
shell-path: /bin/sh
feature: fsmonitor--daemon
libcurl: 8.4.0
zlib: 1.2.12
uname: Darwin 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30
PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6030 arm64
compiler info: clang: 15.0.0 (clang-1500.3.9.4)
libc info: no libc information available
$SHELL (typically, interactive shell): /bin/zsh


[Enabled Hooks]
not run from a git repository - no hooks to show

Re: git commit hangs when run concurrently on macos

[Eric Sunshine]

On Mon, Sep 30, 2024 at 4:58 PM Bennet Huber <bennet.huber@gmail.com> wrote:
> After running this, wait for output to subside, then run `jobs`.
> On my machine, I usually see 5-10 hung "git commit" jobs.
>
> > 3:39:04.020533 run-command.c:758       trace: start_command: /opt/homebrew/opt/git/libexec/git-core/git fsmonitor--daemon run --detach --ipc-threads=8
>
> Setting `USE_FS_MONITOR=false` in the code above consistently results
> in no hung "git commit" commands, which leads me to suspect the
> problem has to do with the fsmonitor feature.  It's also a decent,
> though difficult to discover, workaround.

I don't use fsmonitor and haven't particularly been following the
issue, but there are some active threads on the mailing list regarding
fsmonitor hanging on macOS. Perhaps they are relevant to your case.

https://lore.kernel.org/git/20240930194600.GA1805534@coredump.intra.peff.net/T/

https://lore.kernel.org/git/20240509162219.GA1707955@coredump.intra.peff.net/T/

https://lore.kernel.org/git/pull.1802.git.1727577690390.gitgitgadget@gmail.com/T/