Re: [LSF/MM/BPF TOPIC] Enabling change notification for network and cluster fs

Re: [LSF/MM/BPF TOPIC] Enabling change notification for network and cluster fs

[Steve French]

On Fri, Feb 25, 2022 at 7:49 AM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Fri, Feb 25, 2022 at 08:23:20AM -0500, Vivek Goyal wrote:
> > What about local events. I am assuming you want to supress local events
> > and only deliver remote events. Because having both local and remote
> > events delivered at the same time will be just confusing at best.
>
> This paragraph confuses me.  If I'm writing, for example, a file manager
> and I want it to update its display automatically when another task alters
> the contents of a directory, I don't care whether the modification was
> done locally or remotely.
>
> If I understand the SMB protocol correctly, it allows the client to take
> out a lease on a directory and not send its modifications back to the
> server until the client chooses to (or the server breaks the lease).
> So you wouldn't get any remote notifications because the client hasn't
> told the server.

Directory leases would be broken by file create so the more important
question is what happens when client 1 has a change notification on writes
to files in a directory then client 2 opens a file in the same directory and is
granted a file lease and starts writing to the file (which means the
writes could get cached).   This is probably a minor point because when
writes get flushed from client 2, client 1 (and any others with notifications
requested) will get notified of the event (changes to files in a directory
that they are watching).

Local applications watching a file on a network or cluster mount in Linux
(just as is the case with Windows, Macs etc.) should be able to be notified of
local (cached) writes to a remote file or remote writes to the file from another
client.  I don't think the change is large, and there was an earlier version of
a patch circulated for this

-- 
Thanks,

Steve

Re: [LSF/MM/BPF TOPIC] Enabling change notification for network and cluster fs

[Vivek Goyal]

On Fri, Feb 25, 2022 at 09:27:55AM -0600, Steve French wrote:
> On Fri, Feb 25, 2022 at 7:49 AM Matthew Wilcox <willy@infradead.org> wrote:
> >
> > On Fri, Feb 25, 2022 at 08:23:20AM -0500, Vivek Goyal wrote:
> > > What about local events. I am assuming you want to supress local events
> > > and only deliver remote events. Because having both local and remote
> > > events delivered at the same time will be just confusing at best.
> >
> > This paragraph confuses me.  If I'm writing, for example, a file manager
> > and I want it to update its display automatically when another task alters
> > the contents of a directory, I don't care whether the modification was
> > done locally or remotely.
> >
> > If I understand the SMB protocol correctly, it allows the client to take
> > out a lease on a directory and not send its modifications back to the
> > server until the client chooses to (or the server breaks the lease).
> > So you wouldn't get any remote notifications because the client hasn't
> > told the server.
> 
> Directory leases would be broken by file create so the more important
> question is what happens when client 1 has a change notification on writes
> to files in a directory then client 2 opens a file in the same directory and is
> granted a file lease and starts writing to the file (which means the
> writes could get cached).   This is probably a minor point because when
> writes get flushed from client 2, client 1 (and any others with notifications
> requested) will get notified of the event (changes to files in a directory
> that they are watching).
> 
> Local applications watching a file on a network or cluster mount in Linux
> (just as is the case with Windows, Macs etc.) should be able to be notified of
> local (cached) writes to a remote file or remote writes to the file from another
> client.  I don't think the change is large, and there was an earlier version of
> a patch circulated for this

So local notifications are generated by filesystem code as needed?

IOW, in general disable all local events and let filesystems decide which
local events to generate? And locally cached write is one such example?

Thanks
Vivek

Re: [Lsf-pc] [LSF/MM/BPF TOPIC] Enabling change notification for network and cluster fs

[Amir Goldstein]

On Fri, Feb 25, 2022 at 8:11 PM Steve French <smfrench@gmail.com> wrote:
>
> > IOW, in general disable all local events and let filesystems decide which
> local events to generate? And locally cached write is one such example?
>
> The fs doesn't see cached writes so probably best to still use the common
> existing code for notification on local writes
>

I guess SMB protocol does not allow client B to request a NOTIFY on change
when client A has a directory lease, because requesting NOTIFY requires
getting a read file handle on the dir?

Effectively, smb client needs to open the remote directory for read in order
to prove that the client has read access to the directory, which is the
prerequisite for getting directory change notifications.

The local check for permissions is not enough for remote notifications:
        /* you can only watch an inode if you have read permissions on it */
        error = path_permission(path, MAY_READ);

Thanks,
Amir.

Re: [LSF/MM/BPF TOPIC] Enabling change notification for network and cluster fs

[Vivek Goyal]

On Fri, Feb 25, 2022 at 01:11:10PM -0500, Steve French wrote:
> > IOW, in general disable all local events and let filesystems decide which
> local events to generate? And locally cached write is one such example?
> 
> The fs doesn't see cached writes so probably best to still use the common
> existing code for notification on local writes

Filesystems could provide a hook/function which local notifications could
call and then filesystem could take a decision whether to allow that
particular local event. 

For example, Ioannis implemented inode operation ->fsnotify_event()

https://lore.kernel.org/linux-fsdevel/20211025204634.2517-6-iangelak@redhat.com/

This gives local event to fuse. And now fuse can decide whether to
dispatch the event to user space by calling __fsnotify() or drop it
on the floor.

As of now we drop everything on the floor if remote notifications are
enabled. But I realize that cache=writeback mode can cache writes, so
fs could selectively allow some local events.

I guess for virtiofs, we could allow all local events. Supress remote
events if event is triggered due to the action of same client. If we
use fanotify on server, then it will report PID of process that caused
the event. And if PID belongs to this virtiofsd instance, it probably
could supress remote event. 

That way an application running inside the guest will see all local
events + remote events triggered by other clients. (But should not
see duplicate events).

Thanks
Vivek

> 
> On Fri, Feb 25, 2022, 11:35 Vivek Goyal <vgoyal@redhat.com> wrote:
> 
> > On Fri, Feb 25, 2022 at 09:27:55AM -0600, Steve French wrote:
> > > On Fri, Feb 25, 2022 at 7:49 AM Matthew Wilcox <willy@infradead.org>
> > wrote:
> > > >
> > > > On Fri, Feb 25, 2022 at 08:23:20AM -0500, Vivek Goyal wrote:
> > > > > What about local events. I am assuming you want to supress local
> > events
> > > > > and only deliver remote events. Because having both local and remote
> > > > > events delivered at the same time will be just confusing at best.
> > > >
> > > > This paragraph confuses me.  If I'm writing, for example, a file
> > manager
> > > > and I want it to update its display automatically when another task
> > alters
> > > > the contents of a directory, I don't care whether the modification was
> > > > done locally or remotely.
> > > >
> > > > If I understand the SMB protocol correctly, it allows the client to
> > take
> > > > out a lease on a directory and not send its modifications back to the
> > > > server until the client chooses to (or the server breaks the lease).
> > > > So you wouldn't get any remote notifications because the client hasn't
> > > > told the server.
> > >
> > > Directory leases would be broken by file create so the more important
> > > question is what happens when client 1 has a change notification on
> > writes
> > > to files in a directory then client 2 opens a file in the same directory
> > and is
> > > granted a file lease and starts writing to the file (which means the
> > > writes could get cached).   This is probably a minor point because when
> > > writes get flushed from client 2, client 1 (and any others with
> > notifications
> > > requested) will get notified of the event (changes to files in a
> > directory
> > > that they are watching).
> > >
> > > Local applications watching a file on a network or cluster mount in Linux
> > > (just as is the case with Windows, Macs etc.) should be able to be
> > notified of
> > > local (cached) writes to a remote file or remote writes to the file from
> > another
> > > client.  I don't think the change is large, and there was an earlier
> > version of
> > > a patch circulated for this
> >
> > So local notifications are generated by filesystem code as needed?
> >
> > IOW, in general disable all local events and let filesystems decide which
> > local events to generate? And locally cached write is one such example?
> >
> > Thanks
> > Vivek
> >
> >

Re: [Lsf-pc] [LSF/MM/BPF TOPIC] Enabling change notification for network and cluster fs

[Ralph Boehme]

[-- Attachment #1.1: Type: text/plain, Size: 332 bytes --]

Hi Amir,

On 2/26/22 11:22, Amir Goldstein wrote:
> I guess SMB protocol does not allow client B to request a NOTIFY on change
> when client A has a directory lease, because requesting NOTIFY requires
> getting a read file handle on the dir?
fwiw, you don't get a "W" SMB3 directory lease, so this is not a problem.

-slow

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]