Option to allow fsmonitor to run against repos on network file systems

Option to allow fsmonitor to run against repos on network file systems

[Eric D]

I can appreciate the concerns expressed here:
https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200

However, in my environment, our file servers are very capable and have
the requisite support. It would be great if there was an option to
override this check and allow fsmonitor to operate against network
filesystems.

-Eric

Re: Option to allow fsmonitor to run against repos on network file systems

[Jeff Hostetler]

On 6/30/22 1:11 PM, Eric D wrote:
> I can appreciate the concerns expressed here:
> https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200
> 
> However, in my environment, our file servers are very capable and have
> the requisite support. It would be great if there was an option to
> override this check and allow fsmonitor to operate against network
> filesystems.

Yeah, I was just being cautious.  I probably should have also added
concerns on the remote system being an actual Windows server or a
non-Windows host running SAMBA.  There were just too many combinations
for me to be comfortable enabling it by default (on the initial
release, at least).

Also, the ReadDirectoryChangesW() API limits the buffer size to 64k
for remote handles (because of protocol limitations), so there _may_
be more of an opportunity for dropped events on very busy remote file
systems.  (I never saw any dropped events in my testing (without
intentionally breaking things), but it is a possible concern, so again,
caution and safety...)  And I do handle dropped events and force a
resync and send the client a "trivial" response (so it must do a regular
scan), so output is still correct, but slower.


Having said all of that, I did do lots of testing and never had an
issue with remote drives actually working correctly, so I think it'd
be fine allow a config setting to optionally allow it.  I just didn't
want to clutter up things in advance if no one actually wanted to
use it on remote file systems.


I think it would be fine to have a "fsmonitor.allowRemote" or
"fsmonitor.allowWindowsRemote" config setting and default them to false
for now.  Or until we learn which combinations of remote mounts are
safe and/or problematic.

Jeff

Re: Option to allow fsmonitor to run against repos on network file systems

[Junio C Hamano]

Jeff Hostetler <git@jeffhostetler.com> writes:

> Having said all of that, I did do lots of testing and never had an
> issue with remote drives actually working correctly, so I think it'd
> be fine allow a config setting to optionally allow it.  I just didn't
> want to clutter up things in advance if no one actually wanted to
> use it on remote file systems.
>
> I think it would be fine to have a "fsmonitor.allowRemote" or
> "fsmonitor.allowWindowsRemote" config setting and default them to false
> for now.  Or until we learn which combinations of remote mounts are
> safe and/or problematic.

How about getting rid of "is this remote?" check altogether (which
presumably would simplify the logic) and make it totally up to the
user of the repository?  fsmonitor.disableInRepository that is set
in ~/.gitignore and lists the paths to the repositories (like
safe.directory does), for which fsmonitor gets disabled, may be a
handy mechanism to set up the default (and it can be re-enabled with
per-repository core.fsmonitor).

Re: Option to allow fsmonitor to run against repos on network file systems

[Eric D]

One more possibility:

Leave the check, but make it a warning if there are still concerns
about running fsmonitor against network file systems. Maybe also
provide an option to suppress the warning? Not that much different
from having "fsmonitor.allowRemote" I suppose other than by default
fsmonitor would "just work" for network mounts.

On Fri, Jul 1, 2022 at 2:41 PM Junio C Hamano <gitster@pobox.com> wrote:
>
> Jeff Hostetler <git@jeffhostetler.com> writes:
>
> > Having said all of that, I did do lots of testing and never had an
> > issue with remote drives actually working correctly, so I think it'd
> > be fine allow a config setting to optionally allow it.  I just didn't
> > want to clutter up things in advance if no one actually wanted to
> > use it on remote file systems.
> >
> > I think it would be fine to have a "fsmonitor.allowRemote" or
> > "fsmonitor.allowWindowsRemote" config setting and default them to false
> > for now.  Or until we learn which combinations of remote mounts are
> > safe and/or problematic.
>
> How about getting rid of "is this remote?" check altogether (which
> presumably would simplify the logic) and make it totally up to the
> user of the repository?  fsmonitor.disableInRepository that is set
> in ~/.gitignore and lists the paths to the repositories (like
> safe.directory does), for which fsmonitor gets disabled, may be a
> handy mechanism to set up the default (and it can be re-enabled with
> per-repository core.fsmonitor).
>
>

Re: Option to allow fsmonitor to run against repos on network file systems

[Eric D]

On Fri, Jul 1, 2022 at 3:15 PM Eric D <eric.decosta@gmail.com> wrote:
>
> One more possibility:
>
> Leave the check, but make it a warning if there are still concerns
> about running fsmonitor against network file systems. Maybe also
> provide an option to suppress the warning? Not that much different
> from having "fsmonitor.allowRemote" I suppose other than by default
> fsmonitor would "just work" for network mounts.
>
> On Fri, Jul 1, 2022 at 2:41 PM Junio C Hamano <gitster@pobox.com> wrote:
> >
> > Jeff Hostetler <git@jeffhostetler.com> writes:
> >
> > > Having said all of that, I did do lots of testing and never had an
> > > issue with remote drives actually working correctly, so I think it'd
> > > be fine allow a config setting to optionally allow it.  I just didn't
> > > want to clutter up things in advance if no one actually wanted to
> > > use it on remote file systems.
> > >
> > > I think it would be fine to have a "fsmonitor.allowRemote" or
> > > "fsmonitor.allowWindowsRemote" config setting and default them to false
> > > for now.  Or until we learn which combinations of remote mounts are
> > > safe and/or problematic.
> >
> > How about getting rid of "is this remote?" check altogether (which
> > presumably would simplify the logic) and make it totally up to the
> > user of the repository?  fsmonitor.disableInRepository that is set
> > in ~/.gitignore and lists the paths to the repositories (like
> > safe.directory does), for which fsmonitor gets disabled, may be a
> > handy mechanism to set up the default (and it can be re-enabled with
> > per-repository core.fsmonitor).
> >
> >

After modifying the code and playing with it for a bit, I think simply
removing the check (but still being able to report on it if tracing is
enabled) is the way to go. Keep it simple and if new use cases arrive,
deal with them then.

-Eric

Re: Option to allow fsmonitor to run against repos on network file systems

[Eric D]

On Fri, Jul 1, 2022 at 9:32 AM Jeff Hostetler <git@jeffhostetler.com> wrote:
>
>
>
> On 6/30/22 1:11 PM, Eric D wrote:
> > I can appreciate the concerns expressed here:
> > https://github.com/git/git/commit/d989b266c1a7ef47f27cec75e90f3dfefbfa0200
> >
> > However, in my environment, our file servers are very capable and have
> > the requisite support. It would be great if there was an option to
> > override this check and allow fsmonitor to operate against network
> > filesystems.
>
> Yeah, I was just being cautious.  I probably should have also added
> concerns on the remote system being an actual Windows server or a
> non-Windows host running SAMBA.  There were just too many combinations
> for me to be comfortable enabling it by default (on the initial
> release, at least).
>
> Also, the ReadDirectoryChangesW() API limits the buffer size to 64k
> for remote handles (because of protocol limitations), so there _may_
> be more of an opportunity for dropped events on very busy remote file
> systems.  (I never saw any dropped events in my testing (without
> intentionally breaking things), but it is a possible concern, so again,
> caution and safety...)  And I do handle dropped events and force a
> resync and send the client a "trivial" response (so it must do a regular
> scan), so output is still correct, but slower.
>
>
> Having said all of that, I did do lots of testing and never had an
> issue with remote drives actually working correctly, so I think it'd
> be fine allow a config setting to optionally allow it.  I just didn't
> want to clutter up things in advance if no one actually wanted to
> use it on remote file systems.
>
>
> I think it would be fine to have a "fsmonitor.allowRemote" or
> "fsmonitor.allowWindowsRemote" config setting and default them to false
> for now.  Or until we learn which combinations of remote mounts are
> safe and/or problematic.
>
> Jeff

OK, based on this and other conversations, I have implemented the following:

1. Introduced a new config setting, "fsmonitor.allowRemote"
"fsmonitor.allowRemote" has a default value of false. Setting it to
true overrides
fsmonitor's default behavior of rejecting network-mounted repos.

2. Restricted allowing remote repos to Windows clients using SMB
If the client is not using SMB and using a network path, then
fsmonitor will reject
the repo path regardless of the value of "fsmonitor.allowRemote"

-Eric