Directory notification problem

Directory notification problem

[Alex Larsson]

I discovered a problem with the dnotify API while fixing a FAM bug today.

The problem occurs when you want to watch a file in a directory, and that 
file is changed several times in the same second. When I get the directory 
notify signal on the directory I need to stat the file to see if the 
change was actually in the file. If the file already changed in the 
current second the stat() result will be identical to the previous stat() 
call, since the resolution of mtime and ctime is one second. 

This leads to missed notifications, leaving clients (such as Nautilus or 
Konqueror) displaying an state not representing the current state.

The only userspace solutions I see is to delay all change notifications to 
the end of the second, so that clients always read the correct state. This 
is somewhat countrary to the idea of FAM though, as it does not give 
instant feedback.

Is there any possibility of extending struct stat with a generation 
counter? Or is there another solution to this problem?

/ Alex

Please CC any reply to me, i'm not on the list.

Re: Directory notification problem

[Padraig Brady]

I think if you want better than 1s resolution you need to
handle it in user space. I've done this using 2 methods
outlined below. This first may be more responsive for
your app, though depending on the processing you have
to do on each file it may be too great an overhead to
process things twice, and if you want to process the data
multiple times within a second then you're screwed (you
probably don't want to do this from what I understand?).

A generation counter like you suggest would be very nice though!

Padraig.

/////////////////////////////////////////////////
// Delayed processing method
/////////////////////////////////////////////////
struct file_data
{
    char* name;
    time_t last_mtime;
    long generation_count; //wouldn't this be nice ;-)
}

bool filechanged(const file_data* file)
{
    struct stat stat_st;
    bool modified = false;

    if (!stat(file->name, &stat_st)) {
        if (stat_st.st_mtime != file->last_mtime) {
            modified = true;
            file->last_mtime = stat_st.st_mtime;
            sleep(1); //make sure any new updates in this mtime processed.
        }
    }
    return modified;
}
/////////////////////////////////////////////////
// Double check method
/////////////////////////////////////////////////
struct file_data
{
    char* name;
    time_t last_mtime;
    bool secondRun = false;
    long generation_count; //wouldn't this be nice ;-)
}

bool filechanged(const file_data* file)
{
    struct stat stat_st;
    bool modified = false;

    if (!stat(file->name, &stat_st)) {
        if ((stat_st.st_mtime != file->last_mtime) || (file->secondRun 
== true)) {
            modified = true;
            /* make sure don't miss data for this mtime (1 second 
resolution).
               Will do redundant check if no data written between now 
and next check
               but at least no new data will go unnoticed. Note this 
function must be called
               at a resolution >= 1 second for this to work. */

            if (file->last_mtime != stat_st.st_mtime) {
                    file->last_mtime = stat_st.st_mtime;
                    file->secondRun = true; //reset
            } else {
                    file->secondRun = false;
            }
        }
    }
    return modified;
}

Alex Larsson wrote:

>I discovered a problem with the dnotify API while fixing a FAM bug today.
>
>The problem occurs when you want to watch a file in a directory, and that 
>file is changed several times in the same second. When I get the directory 
>notify signal on the directory I need to stat the file to see if the 
>change was actually in the file. If the file already changed in the 
>current second the stat() result will be identical to the previous stat() 
>call, since the resolution of mtime and ctime is one second. 
>
>This leads to missed notifications, leaving clients (such as Nautilus or 
>Konqueror) displaying an state not representing the current state.
>
>The only userspace solutions I see is to delay all change notifications to 
>the end of the second, so that clients always read the correct state. This 
>is somewhat countrary to the idea of FAM though, as it does not give 
>instant feedback.
>
>Is there any possibility of extending struct stat with a generation 
>counter? Or is there another solution to this problem?
>
>/ Alex
>
>Please CC any reply to me, i'm not on the list.
>

Re: Directory notification problem

[Pavel Machek]

Hi!

> I discovered a problem with the dnotify API while fixing a FAM bug today.
> 
> The problem occurs when you want to watch a file in a directory, and that 
> file is changed several times in the same second. When I get the directory 

Does this mean that we have notification API in Linus' tree?
									Pavel

Re: Directory notification problem

[Alex Larsson]

On Fri, 5 Oct 2001, Pavel Machek wrote:

> Hi!
> 
> > I discovered a problem with the dnotify API while fixing a FAM bug today.
> > 
> > The problem occurs when you want to watch a file in a directory, and that 
> > file is changed several times in the same second. When I get the directory 
> 
> Does this mean that we have notification API in Linus' tree?

Yes. since a 2.3.xx somewhere.

/ Alex