Re: inotify limits - thousands (tens of thousands?) of watches

[Tvrtko Ursulin <tvrtko.ursulin@sophos.com>]

On Wednesday 20 May 2009 10:26:57 Marcin Krol wrote:
> Hello everyone,
>
> First, apols for using up bandwidth, but I honestly found no other place
> where I can ask about this (and get meaningful reply).
>
> I'm not a kernel programmer, but I want to develop a program that would
> watch modifications in *all* user directories on a busy server using
> inotify.
>
> This is for high-availability purposes - events would be collected and
> once every several minutes changed dirs would be rsync'ed to failover
> server or smth like that would be done.
>
> As inotify watches particular directory and not its subdirs, I would
> have to watch all directories.
>
> This means I would have to create thousands or even tens of thousands of
> inotify watches.
>
> So my questions are:
>
> 1. is it safe? that is, will it not lock the kernel up, or cause
> excessive memory consumption?
>
> 2. is it economic in terms of CPU time and RAM? I have no idea how to
> even measure such a thing happening in the kernel..

Hi,

I don't know the specific answers apart my intuitive feeling, which is that it 
probably would cause excessive memory use and that it is not elegant at all.

What may be interesting to you is the notification "engine" rewrite Eric Paris 
is doing and new fanotify interface which will, hopefully, come soon on top 
of it. With fanotify you will be able to monitor all changes much more easily 
and efficiently.

Have a look at patch descriptions in:
http://people.redhat.com/~eparis/fsnotify/

Tvrtko