Callbacks to userspace from VFS ?

Callbacks to userspace from VFS ?

[Martin Rode]

Dear kernel hackers,

after programming at least 10 scripts polling a what we call
"hot-folder" for new files I had the idea to integrate call backs into
the file system layer of the linux kernel.

I would like to tell the kernel to callback my program whenever a file
or directory is being inserted, updated or deleted.

A simple approach could look like this (from the users POV):

mount -o callback=/tmp/myprogram callback_options=some_options
callback_folder=hotfolder callback_folder=hotfolder2 /dev/some /home

depending on what has happend in "hotfolder" or "hotfolder2" the
"myprogram" would be started and receive the two arguments:

DELETE filename of the file deleted
UPDATE filename
INSERT filename

It would be neat if one could change the mount options while the
filesystem is mounted.

If you could implement such a feature we had another great argument why
the linux kernel has something to offer which others haven't. With such
a feature one could program solutions for the real world which are
always annoying to program (cue: "hotfolder"!).

What do you people think about the idea? Please reply to me personally,
too, I'm not a subscriber.

Thanks for taking a look at the idea.

;Martin



-- 
                  Dipl.-Kfm. Martin Rode                    
                     Managing Director

                martin.rode@programmfabrik.de                  

Programmfabrik GmbH                 Fon +49-(0)30-4281-8001
Frankfurter Allee 73d               Fax +49-(0)30-4281-8008
10247 Berlin                        Funk +49-(0)163-5321400
 
                http://www.programmfabrik.de/
 

Re: Callbacks to userspace from VFS ?

[Martin Rode]

Hi Nikita,

I did not look deeper into F_NOTIFY, but if I only get a SIGNAL I don't
know _what_ has happened (or better on what file something has
happened). But to process the new / updated files I need the filename.
If I only learn which directory was updated I still have to find out
_which_ file is new or was changed.

Regards,

;Martin

-- 
                  Dipl.-Kfm. Martin Rode                    
                     Managing Director

                martin.rode@programmfabrik.de                  

Programmfabrik GmbH                 Fon +49-(0)30-4281-8001
Frankfurter Allee 73d               Fax +49-(0)30-4281-8008
10247 Berlin                        Funk +49-(0)163-5321400
 
                http://www.programmfabrik.de/
 

Re: Callbacks to userspace from VFS ?

[Alex Riesen]

Hi,

try reading Documentation/dnotify.txt. Maybe it'll be enough for you.

-alex

On Wed, Apr 17, 2002 at 04:21:13PM +0200, Martin Rode wrote:
> Dear kernel hackers,
> 
> after programming at least 10 scripts polling a what we call
> "hot-folder" for new files I had the idea to integrate call backs into
> the file system layer of the linux kernel.
> 
> I would like to tell the kernel to callback my program whenever a file
> or directory is being inserted, updated or deleted.
> 
> A simple approach could look like this (from the users POV):
> 
> mount -o callback=/tmp/myprogram callback_options=some_options
> callback_folder=hotfolder callback_folder=hotfolder2 /dev/some /home
> 
> depending on what has happend in "hotfolder" or "hotfolder2" the
> "myprogram" would be started and receive the two arguments:
> 
> DELETE filename of the file deleted
> UPDATE filename
> INSERT filename
> 
> It would be neat if one could change the mount options while the
> filesystem is mounted.
> 
> If you could implement such a feature we had another great argument why
> the linux kernel has something to offer which others haven't. With such
> a feature one could program solutions for the real world which are
> always annoying to program (cue: "hotfolder"!).
> 
> What do you people think about the idea? Please reply to me personally,
> too, I'm not a subscriber.
> 
> Thanks for taking a look at the idea.
> 
> ;Martin

Re: Callbacks to userspace from VFS ?

[Jamie Lokier]

Martin Rode wrote:
> I did not look deeper into F_NOTIFY, but if I only get a SIGNAL I don't
> know _what_ has happened (or better on what file something has
> happened). But to process the new / updated files I need the filename.
> If I only learn which directory was updated I still have to find out
> _which_ file is new or was changed.

If you use a real-time signal (number >= SIGRTMIN), then you can find
out which directory was affected.  Look for "siginfo" and SA_SIGINFO.

But you are right that it doesn't tell you which filename is new or was
changed.

-- Jamie

Re: Callbacks to userspace from VFS ?

[Keith Owens]

On 17 Apr 2002 16:21:13 +0200, 
Martin Rode <martin.rode@programmfabrik.de> wrote:
>after programming at least 10 scripts polling a what we call
>"hot-folder" for new files I had the idea to integrate call backs into
>the file system layer of the linux kernel.
>
>I would like to tell the kernel to callback my program whenever a file
>or directory is being inserted, updated or deleted.

dnotify already exists, although you have to work out what has changed.

XFS implements DMAPI (Data Management API) event callouts which give
much more details.  DMAPI is designed for full blown Hierarchical
Storage Managements systems.
http://www.opengroup.org/pubs/catalog/c429.htm to purchase the DMAPI
standard, there is also a free (with registration) online standard.

Not speaking for SGI.

Re: Callbacks to userspace from VFS ?

[Michael Clark]

fam and imon also sound like they would do what you want to do.

fam works with or without the imon (inode monitor) kernel extension.
If imon is present, then fam doesn't need to poll for changes on interested
files/directories and operation is much more efficient. I think imon still
needs some work from reading the docs (is only version 0.0.2).

~mc

http://oss.sgi.com/projects/fam/

Keith Owens wrote:

>On 17 Apr 2002 16:21:13 +0200, 
>Martin Rode <martin.rode@programmfabrik.de> wrote:
>
>>after programming at least 10 scripts polling a what we call
>>"hot-folder" for new files I had the idea to integrate call backs into
>>the file system layer of the linux kernel.
>>
>>I would like to tell the kernel to callback my program whenever a file
>>or directory is being inserted, updated or deleted.
>>
>
>dnotify already exists, although you have to work out what has changed.
>
>XFS implements DMAPI (Data Management API) event callouts which give
>much more details.  DMAPI is designed for full blown Hierarchical
>Storage Managements systems.
>http://www.opengroup.org/pubs/catalog/c429.htm to purchase the DMAPI
>standard, there is also a free (with registration) online standard.
>
>Not speaking for SGI.
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>