Re: File monitor problem

[Amir Goldstein <amir73il@gmail.com>]

On Tue, Dec 24, 2019 at 5:49 AM Amir Goldstein <amir73il@gmail.com> wrote:
>
> > > I can see the need for FAN_DIR_MODIFIED_WITH_NAME
> > > (stupid name, I know) - generated when something changed with names in a
> > > particular directory, reported with FID of the directory and the name
> > > inside that directory involved with the change. Directory watching
> > > application needs this to keep track of "names to check". Is the name
> > > useful with any other type of event? _SELF events cannot even sensibly have
> > > it so no discussion there as you mention below. Then we have OPEN, CLOSE,
> > > ACCESS, ATTRIB events. Do we have any use for names with those?
> > >
> >
> > The problem is that unlike dir fid, file fid cannot be reliably resolved
> > to path, that is the reason that I implemented  FAN_WITH_NAME
> > for events "possible on child" (see branch fanotify_name-wip).
> >
> > A filesystem monitor typically needs to be notified on name changes and on
> > data/metadata modifications.
> >

And just before 2019 ends, here is the promised demo.

The kernel branch was added support for events "on child" with name [1].
The inotifywatch demo created for FAN_REPORT_FID was extended to
watch events with FAN_REPORT_FID_NAME [2].

[1] https://github.com/amir73il/linux/commits/fanotify_name
[2] https://github.com/amir73il/inotify-tools/commits/fanotify_name

The demo branch includes the script test_demo.sh, whose output can be
seen here below, that does:

1. Create a small data set (in a filesystem mounted at /vdf)
2. Set up an fanotify filesystem mark watching for all fs changes
3. Make some filesystem changes on files and dirs
4. Read events with fid+name info after 2 seconds delay
5. Check the uptodate path of events with open_by_handle_at+faccessat
6. Print summary of all paths that require "attention"
7. Paths that are currently ENOENT are marked with "(deleted)"

The report includes all the information needed to sync filesystem
changes to mirror or to re-index filesystem after changes.

For example, the file a/b/c/1 was moved to a/b/c/d/e/f/g/1 and then
directory a/b/c/d/e/f/g was moved to a/b/c/d/e/G.
In the report, the paths a/b/c/1 and a/b/c/d/e/f/g are listed as (deleted)
and the paths a/b/c/d/e/G and a/b/c/d/e/G/1 are listed as changed.
There is no record in the report of the intermediate path a/b/c/d/e/f/g/1.

Happy new year!
Amir.

---------------
# ./test_demo.sh /vdf
+ WD=/vdf
+ cd /vdf
+ rm -rf a
+ mkdir -p a/b/c/d/e/f/g/
+ touch a/b/c/0 a/b/c/1 a/b/c/d/e/f/g/0
+ sleep 1
+ inotifywatch --global --writes --timeout -2 /vdf
Establishing filesystem global watch...
Finished establishing watches, now collecting statistics.
Sleeping for 2 seconds...
+
+ t=Create files and dirs...
+ touch a/0 a/1 a/2 a/3
+ mkdir a/dir0 a/dir1 a/dir2
+
+ t=Rename files and dirs...
+ mv a/0 a/3
+ mv a/dir0 a/dir3
+
+ t=Delete files and dirs...
+ rm a/1
+ rmdir a/dir1
+
+ t=Modify files and dirs...
+ chmod +x a/b/c/d
+ touch a/b/c/0
+
+ t=Move files and dirs...
+ mv a/b/c/1 a/b/c/d/e/f/g/1
+ mv a/b/c/d/e/f/g a/b/c/d/e/G
+
[fid=fd50.0.2007402;name='0'] /vdf/a/0 (deleted)
[fid=fd50.0.2007402;name='1'] /vdf/a/1 (deleted)
[fid=fd50.0.2007402;name='2'] /vdf/a/2
[fid=fd50.0.2007402;name='3'] /vdf/a/3
[fid=fd50.0.2007402;name='dir0'] /vdf/a/dir0 (deleted)
[fid=fd50.0.2007402;name='dir1'] /vdf/a/dir1 (deleted)
[fid=fd50.0.2007402;name='dir2'] /vdf/a/dir2
[fid=fd50.0.2007402;name='dir3'] /vdf/a/dir3
[fid=fd50.0.8c;name='d'] /vdf/a/b/c/d
[fid=fd50.0.8c;name='0'] /vdf/a/b/c/0
[fid=fd50.0.8c;name='1'] /vdf/a/b/c/1 (deleted)
[fid=fd50.0.2007403;name='1'] /vdf/a/b/c/d/e/G/1
[fid=fd50.0.10000c2;name='g'] /vdf/a/b/c/d/e/f/g (deleted)
[fid=fd50.0.2007403;name='G'] /vdf/a/b/c/d/e/G
total  filename
2      /vdf/a/0 (deleted)
2      /vdf/a/1 (deleted)
2      /vdf/a/3
2      /vdf/a/dir0 (deleted)
2      /vdf/a/dir1 (deleted)
1      /vdf/a/2
1      /vdf/a/dir2
1      /vdf/a/dir3
1      /vdf/a/b/c/d
1      /vdf/a/b/c/0
1      /vdf/a/b/c/1 (deleted)
1      /vdf/a/b/c/d/e/G/1
1      /vdf/a/b/c/d/e/f/g (deleted)
1      /vdf/a/b/c/d/e/G