peculiar disappearance of most audit rules

peculiar disappearance of most audit rules

[Peter Grandi]

Hi, I have started using 'auditd', mostly to monitor various directories
where packages get installed to check for changes in their contents,
with rules like:

  -w /bin                         -p wa -k pkg-s
  -w /boot                        -p wa -k pkg-s
  -w /etc                         -p wa -k pkg-s
  -w /lib                         -p wa -k pkg-s
  -w /lib32                       -p wa -k pkg-s
  -w /lib64                       -p wa -k pkg-s
  -w /opt                         -p wa -k pkg-s
  -w /usr                         -p wa -k pkg-s

  -w /fs/sozan/loc                -p wa -k pkg-l
  -w /fs/sozan/loc32-el5          -p wa -k pkg-l
  -w /fs/sozan/loc64-u12          -p wa -k pkg-l
  -w /fs/sozan/com                -p wa -k pkg-l
  -w /fs/sozan/com32-el5          -p wa -k pkg-l
  -w /fs/sozan/com64-u12          -p wa -k pkg-l

After setting them, I can verify that for example creating, updating and
deleting a file in '/boot' or '/opt' gets reported.

Wheat then happens is that even if I set 'auditctl -e 2' some of the
rules disappear, usually at around the same time as 'cron.daily' scripts
run, and some more disappear later. This usually seems to relate to
times where there some significant IO activity ('mlocate' scan, backup),
but this is a guess.

For example:

  time->Thu Apr 17 07:58:44 2014
  type=CONFIG_CHANGE msg=audit(1397717924.255:37148): op="remove rule" dir="/boot" key="pkg-s" list=4 res=1
  time->Thu Apr 17 07:59:04 2014
  type=CONFIG_CHANGE msg=audit(1397717944.762:37151): op="remove rule" dir="/opt" key="pkg-s" list=4 res=1
  time->Thu Apr 17 10:01:02 2014
  type=CONFIG_CHANGE msg=audit(1397725262.301:37157): op="remove rule" dir="/fs/sozan/loc64-u12" key="pkg-l" list=4 res=1
  time->Thu Apr 17 10:01:02 2014
  type=CONFIG_CHANGE msg=audit(1397725262.301:37156): op="remove rule" dir="/fs/sozan/loc32-el5" key="pkg-l" list=4 res=1

There is no equivalent line in 'dmesg'.

I understand that the 'audit' kernel modules may remove rules if they
refer to invalid paths, but all the relevant directories do exist, as
for example '/boot' and '/opt' are the standard usual directories in the
"root" tree itself:

  $  ls -ldn /boot /opt /fs/sozan/loc64-u12 /fs/sozan/loc32-el5
  drwxr-xr-x 3 0 0 4096 Apr 21 07:22 /boot
  drwxrwsr-x 7 1 1   61 Jul 30  2011 /fs/sozan/loc32-el5
  drwxrwsr-x 5 1 1   39 Oct  4  2011 /fs/sozan/loc64-u12
  drwxr-xr-x 7 0 0 4096 Apr 20 14:52 /opt

  $  df /boot/. /opt/. /fs/sozan/loc64-u12/. /fs/sozan/loc32-el5/.
  Filesystem     1M-blocks  Used Available Use% Mounted on
  /dev/sda3          24815 16853      4106  81% /
  /dev/sda3          24815 16853      4106  81% /
  /dev/sda6          90048 82355      7694  92% /fs/sozan
  /dev/sda6          90048 82355      7694  92% /fs/sozan

This is happening on two similarly configured Ubuntu 12.04 systems with
both 3.2 and 3.11 Ubuntu "official" kernels. I also have an AppArmor
configuration which seem to trigger bugs in AppArmor, but all the
relative profiles are essentially unchanged.

Eventually around almost all of the rules I have set "disappear". For
example of all these rules:

  LIST_RULES: exit,always dir=/fs/sozan/search (0x10) perm=r key=pkg-r
  LIST_RULES: exit,always dir=/fs/sozan/mlocate (0x11) perm=r key=pkg-r
  ....
  LIST_RULES: exit,always dir=/bin (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/boot (0x5) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/etc (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/lib (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/lib32 (0x6) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/lib64 (0x6) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/opt (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/usr (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/fs/sozan/loc (0xd) perm=wa key=pkg-l
  LIST_RULES: exit,always dir=/fs/sozan/loc32-el5 (0x13) perm=wa key=pkg-l
  LIST_RULES: exit,always dir=/fs/sozan/loc64-u12 (0x13) perm=wa key=pkg-l
  LIST_RULES: exit,always dir=/fs/sozan/com (0xd) perm=wa key=pkg-l
  LIST_RULES: exit,always dir=/fs/sozan/com32-el5 (0x13) perm=wa key=pkg-l
  LIST_RULES: exit,always dir=/fs/sozan/com64-u12 (0x13) perm=wa key=pkg-l

Only the first two have not "disappeared" on one of the systems.

This is rather peculiar, please let me know if it is a configuration
error, an issue, and any fixes or workaround if available (other than
running 'auditctl -R /etc/audit/audit.rules' every few minutes via CRON).

Re: peculiar disappearance of most audit rules

[Steve Grubb]

On Monday, April 21, 2014 06:49:24 PM Peter Grandi wrote:
> Hi, I have started using 'auditd', mostly to monitor various directories
> where packages get installed to check for changes in their contents,
> with rules like:
> 
>   -w /bin                         -p wa -k pkg-s
>   -w /boot                        -p wa -k pkg-s
>   -w /etc                         -p wa -k pkg-s
>   -w /lib                         -p wa -k pkg-s
>   -w /lib32                       -p wa -k pkg-s
>   -w /lib64                       -p wa -k pkg-s
>   -w /opt                         -p wa -k pkg-s
>   -w /usr                         -p wa -k pkg-s
> 
>   -w /fs/sozan/loc                -p wa -k pkg-l
>   -w /fs/sozan/loc32-el5          -p wa -k pkg-l
>   -w /fs/sozan/loc64-u12          -p wa -k pkg-l
>   -w /fs/sozan/com                -p wa -k pkg-l
>   -w /fs/sozan/com32-el5          -p wa -k pkg-l
>   -w /fs/sozan/com64-u12          -p wa -k pkg-l
> 
> After setting them, I can verify that for example creating, updating and
> deleting a file in '/boot' or '/opt' gets reported.
> 
> Wheat then happens is that even if I set 'auditctl -e 2' some of the
> rules disappear, usually at around the same time as 'cron.daily' scripts
> run, and some more disappear later. This usually seems to relate to
> times where there some significant IO activity ('mlocate' scan, backup),
> but this is a guess.
> 
> For example:
> 
>   time->Thu Apr 17 07:58:44 2014
>   type=CONFIG_CHANGE msg=audit(1397717924.255:37148): op="remove rule"
> dir="/boot" key="pkg-s" list=4 res=1 time->Thu Apr 17 07:59:04 2014
>   type=CONFIG_CHANGE msg=audit(1397717944.762:37151): op="remove rule"
> dir="/opt" key="pkg-s" list=4 res=1 time->Thu Apr 17 10:01:02 2014
>   type=CONFIG_CHANGE msg=audit(1397725262.301:37157): op="remove rule"
> dir="/fs/sozan/loc64-u12" key="pkg-l" list=4 res=1 time->Thu Apr 17
> 10:01:02 2014
>   type=CONFIG_CHANGE msg=audit(1397725262.301:37156): op="remove rule"
> dir="/fs/sozan/loc32-el5" key="pkg-l" list=4 res=1
> 
> There is no equivalent line in 'dmesg'.
> 
> I understand that the 'audit' kernel modules may remove rules if they
> refer to invalid paths, but all the relevant directories do exist, as
> for example '/boot' and '/opt' are the standard usual directories in the
> "root" tree itself:

What happens is that the text path that you put in a watch is a human 
convenience. The kernel doesn't understand strings, it understands numbers. It 
changes the path into device and inode information. This is, technically, what 
your watch is on. If the inode disappears, then the rule is ejected. Rules can 
survive across renames but not deletions.

I don't know what is managing your system, but its probably deleting paths. 
You might do a ls -i to get the inode number of the directories and then when 
you notice the rules being ejected, re-run the ls -i and see if the inodes 
have changed.

-Steve


>   $  ls -ldn /boot /opt /fs/sozan/loc64-u12 /fs/sozan/loc32-el5
>   drwxr-xr-x 3 0 0 4096 Apr 21 07:22 /boot
>   drwxrwsr-x 7 1 1   61 Jul 30  2011 /fs/sozan/loc32-el5
>   drwxrwsr-x 5 1 1   39 Oct  4  2011 /fs/sozan/loc64-u12
>   drwxr-xr-x 7 0 0 4096 Apr 20 14:52 /opt
> 
>   $  df /boot/. /opt/. /fs/sozan/loc64-u12/. /fs/sozan/loc32-el5/.
>   Filesystem     1M-blocks  Used Available Use% Mounted on
>   /dev/sda3          24815 16853      4106  81% /
>   /dev/sda3          24815 16853      4106  81% /
>   /dev/sda6          90048 82355      7694  92% /fs/sozan
>   /dev/sda6          90048 82355      7694  92% /fs/sozan
> 
> This is happening on two similarly configured Ubuntu 12.04 systems with
> both 3.2 and 3.11 Ubuntu "official" kernels. I also have an AppArmor
> configuration which seem to trigger bugs in AppArmor, but all the
> relative profiles are essentially unchanged.
> 
> Eventually around almost all of the rules I have set "disappear". For
> example of all these rules:
> 
>   LIST_RULES: exit,always dir=/fs/sozan/search (0x10) perm=r key=pkg-r
>   LIST_RULES: exit,always dir=/fs/sozan/mlocate (0x11) perm=r key=pkg-r
>   ....
>   LIST_RULES: exit,always dir=/bin (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/boot (0x5) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/etc (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/lib (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/lib32 (0x6) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/lib64 (0x6) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/opt (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/usr (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/fs/sozan/loc (0xd) perm=wa key=pkg-l
>   LIST_RULES: exit,always dir=/fs/sozan/loc32-el5 (0x13) perm=wa key=pkg-l
>   LIST_RULES: exit,always dir=/fs/sozan/loc64-u12 (0x13) perm=wa key=pkg-l
>   LIST_RULES: exit,always dir=/fs/sozan/com (0xd) perm=wa key=pkg-l
>   LIST_RULES: exit,always dir=/fs/sozan/com32-el5 (0x13) perm=wa key=pkg-l
>   LIST_RULES: exit,always dir=/fs/sozan/com64-u12 (0x13) perm=wa key=pkg-l
> 
> Only the first two have not "disappeared" on one of the systems.
> 
> This is rather peculiar, please let me know if it is a configuration
> error, an issue, and any fixes or workaround if available (other than
> running 'auditctl -R /etc/audit/audit.rules' every few minutes via CRON).
> 
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit

Re: peculiar disappearance of most audit rules

[lists_todd]

[-- Attachment #1.1: Type: text/plain, Size: 429 bytes --]


On Apr 21, 2014, at 11:28 AM, Steve Grubb <sgrubb@redhat.com> wrote:

> What happens is that the text path that you put in a watch is a human 
> convenience. The kernel doesn't understand strings, it understands numbers. It 
> changes the path into device and inode information.

Cool. So I am guessing the rule works even if someone creates a hard link to the same watched path and access files through that other path?

Todd


[-- Attachment #1.2: Type: text/html, Size: 1192 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: peculiar disappearance of most audit rules

[Eric Paris]

On Mon, 2014-04-21 at 11:35 -0700, lists_todd@mac.com wrote:
> 
> On Apr 21, 2014, at 11:28 AM, Steve Grubb <sgrubb@redhat.com> wrote:
> 
> > What happens is that the text path that you put in a watch is a
> > human 
> > convenience. The kernel doesn't understand strings, it understands
> > numbers. It 
> > changes the path into device and inode information.
> 
> 
> Cool. So I am guessing the rule works even if someone creates a hard
> link to the same watched path and access files through that other
> path?

As I remember, and it's been a long time, watches should survive even if
the object being watched is deleted and recreated.  I seemed to remember
it was only if the parent directory is deleted that rules get evicted.

So that doesn't explain it for /boot!  Pretty darn hard to delete /!
But it could easily make sense for your other areas being watched...

But yes, if you watch /etc/shadow and someone accesses that inode
through another hard link, you will get audit records...

Re: peculiar disappearance of most audit rules

[Peter Grandi]

[ ... ]

>> For example:

>> time->Thu Apr 17 07:58:44 2014
>> type=CONFIG_CHANGE msg=audit(1397717924.255:37148): op="remove rule" dir="/boot" key="pkg-s" list=4 res=1
>> time->Thu Apr 17 07:59:04 2014
>> type=CONFIG_CHANGE msg=audit(1397717944.762:37151): op="remove rule" dir="/opt" key="pkg-s" list=4 res=1
>> time->Thu Apr 17 10:01:02 2014

>> type=CONFIG_CHANGE msg=audit(1397725262.301:37157): op="remove rule" dir="/fs/sozan/loc64-u12" key="pkg-l" list=4 res=1
>> time->Thu Apr 17 10:01:02 2014
>> type=CONFIG_CHANGE msg=audit(1397725262.301:37156): op="remove rule" dir="/fs/sozan/loc32-el5" key="pkg-l" list=4 res=1

> [ ... ] device and inode information. This is, technically,
> what your watch is on. If the inode disappears, then the rule
> is ejected. Rules can survive across renames but not
> deletions.

> I don't know what is managing your system, but its probably
> deleting paths.

I am the sole user (as far as I know...) of both systems, and I
am pretty sure I was asleep at least at some of the reported
times, and I can't imagine any of the system scripts deleting
and recreating '/boot' and '/opt', for example. Also I checked
the 'm' times of '/' and '/fs/sozan' and they are a few weeks
old. None of the "disappeared" paths seems to have been modified
in any way.

BTW this has happened also on a far smaller scale on a Debian
7/Wheezy system. Again a system where I am the sysadm and only
user, and it seems roughly at the same time as a treewalk.

The vague impression I have in both cases is that there is some
reason why under high load 'audit' just loses or deletes watches.
Note that this was not under high _logging_ load, because the
watches for '/opt' and '/boot' are to log in case of writing or
attribute changes, and the treewalk is entirely (on one side)
read-only. Indeed I see not even a trace of logging about those
accesses.

Anyhow, I have now recorded the inos of the watched directories,
and I shall also run 'inotifywait -m /' to catch if possible any
changes in '/opt' and '/boot'.

Re: peculiar disappearance of most audit rules

[Peter Grandi]

>> I don't know what is managing your system, but its probably
>> deleting paths.

> I am the sole user (as far as I know...) of both systems, [
> ... ] None of the "disappeared" paths seems to have been
> modified in any way. [ ... ] Anyhow, I have now recorded the
> inos of the watched directories, and I shall also run
> 'inotifywait -m /' to catch if possible any changes in '/opt'
> and '/boot'.

I have done this and this morning during 'mlocate' treewalking
some of the usual paths disappeared; I verified the inos and
the 'inotify' output and no inos changed nor any of the watched
directories changed.

Since the list of directories that *do not* disappear is
usually:

  LIST_RULES: exit,always dir=/bin (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/etc (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/lib (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/usr (0x4) perm=wa key=pkg-s
  LIST_RULES: exit,always dir=/fs/sozan/loc (0xd) perm=wa key=pkg-l
  LIST_RULES: exit,always dir=/fs/sozan/com (0xd) perm=wa key=pkg-l

and those that disappear tend to be far less frequently used
directories like '/boot', '/opt', '/lib32'. Rereading this:

>> [ ... ] device and inode information. This is, technically,
>> what your watch is on. If the inode disappears, then the rule
>> is ejected. Rules can survive across renames but not deletions.

it appears that I misread earlier: this says "inode", not
"inum". Also it says "inode disappears", which is not
necessarily always because the on-disk inode is deleted.

Thus I have come up with a potential explanation:

  * The 'audit' module does not identify the watched file and
    directory by (device,ino) but by a pointer to an inode table
    entry, a bit like a filesystem module would.
  * During treewalks a lot of inodes get cached in the in-memory
    inode table.
  * This creates pressure on the inode tables and thus the least
    used (in some sense) inodes get evicted, and this includes
    those for the "disappearing directories".
  * When these least used inodes are evicted the 'audit' module
    sees it as if it was a removal of the inode.

If the above is the right explanation it is a pretty big deal,
because it means that a way to disable many/most 'audit' watches
on files is to create and access a lot of inodes, which is
pretty easy to do.

Re: peculiar disappearance of most audit rules

[Steve Grubb]

On Tuesday, April 22, 2014 09:53:15 PM Peter Grandi wrote:
> >> I don't know what is managing your system, but its probably
> >> deleting paths.
> > 
> > I am the sole user (as far as I know...) of both systems, [
> > ... ] None of the "disappeared" paths seems to have been
> > modified in any way. [ ... ] Anyhow, I have now recorded the
> > inos of the watched directories, and I shall also run
> > 'inotifywait -m /' to catch if possible any changes in '/opt'
> > and '/boot'.
> 
> I have done this and this morning during 'mlocate' treewalking
> some of the usual paths disappeared; I verified the inos and
> the 'inotify' output and no inos changed nor any of the watched
> directories changed.
> 
> Since the list of directories that *do not* disappear is
> usually:
> 
>   LIST_RULES: exit,always dir=/bin (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/etc (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/lib (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/usr (0x4) perm=wa key=pkg-s
>   LIST_RULES: exit,always dir=/fs/sozan/loc (0xd) perm=wa key=pkg-l
>   LIST_RULES: exit,always dir=/fs/sozan/com (0xd) perm=wa key=pkg-l
> 
> and those that disappear tend to be far less frequently used
> 
> directories like '/boot', '/opt', '/lib32'. Rereading this:
> >> [ ... ] device and inode information. This is, technically,
> >> what your watch is on. If the inode disappears, then the rule
> >> is ejected. Rules can survive across renames but not deletions.
> 
> it appears that I misread earlier: this says "inode", not
> "inum". Also it says "inode disappears", which is not
> necessarily always because the on-disk inode is deleted.
> 
> Thus I have come up with a potential explanation:
> 
>   * The 'audit' module does not identify the watched file and
>     directory by (device,ino) but by a pointer to an inode table
>     entry, a bit like a filesystem module would.
>   * During treewalks a lot of inodes get cached in the in-memory
>     inode table.
>   * This creates pressure on the inode tables and thus the least
>     used (in some sense) inodes get evicted, and this includes
>     those for the "disappearing directories".
>   * When these least used inodes are evicted the 'audit' module
>     sees it as if it was a removal of the inode.
> 
> If the above is the right explanation it is a pretty big deal,

I don't know if that is in fact what happens. But if it is, I would agree with 
your conclusion.

-Steve


> because it means that a way to disable many/most 'audit' watches
> on files is to create and access a lot of inodes, which is
> pretty easy to do.

Re: peculiar disappearance of most audit rules

[Peter Grandi]

[ ... ]

> Thus I have come up with a potential explanation:

>   * The 'audit' module does not identify the watched file and
>     directory by (device,ino) but by a pointer to an inode table
>     entry, a bit like a filesystem module would.

I had a look at the code and it seems it relies on 'inotify' and
the code does get pointers at the relevant in-memory inode
descriptors.

>   * During treewalks a lot of inodes get cached in the in-memory
>     inode table.
>   * This creates pressure on the inode tables and thus the least
>     used (in some sense) inodes get evicted, and this includes
>     those for the "disappearing directories".
>   * When these least used inodes are evicted the 'audit' module
>     sees it as if it was a removal of the inode.

To corroborate this I have been running:

  while true
  do
    for D in $(< audit-names.txt)
    do (cd "$D" && exec sleep 3001)&
    done
    sleep 3001
  done

Which has the effect of marking the relevant directories as the
active current directories of each 'sleep' process, and none of
those directories were "disappeared" from the 'audit' active
rules list.

The 'inotify' code has a comment that claims:

> * inode: Pinned so long as the inode is associated with a watch, from
> * inotify_add_watch() to the final put_inotify_watch().

They use 'igrab'/'iput', and 'audit_tree.c' and 'audit_watch.c'
uses them, so I wonder what is missing.

Re: peculiar disappearance of most audit rules

[Eric Paris]

What's the kernel in question?  audit hasn't used "inotify" in a long
time.  We now use "fsnotify".   but in either case, the inodes aren't
supposed to be able to be kicked out of core...

On Wed, 2014-04-23 at 09:04 +0100, Peter Grandi wrote:
> [ ... ]
> 
> > Thus I have come up with a potential explanation:
> 
> >   * The 'audit' module does not identify the watched file and
> >     directory by (device,ino) but by a pointer to an inode table
> >     entry, a bit like a filesystem module would.
> 
> I had a look at the code and it seems it relies on 'inotify' and
> the code does get pointers at the relevant in-memory inode
> descriptors.
> 
> >   * During treewalks a lot of inodes get cached in the in-memory
> >     inode table.
> >   * This creates pressure on the inode tables and thus the least
> >     used (in some sense) inodes get evicted, and this includes
> >     those for the "disappearing directories".
> >   * When these least used inodes are evicted the 'audit' module
> >     sees it as if it was a removal of the inode.
> 
> To corroborate this I have been running:
> 
>   while true
>   do
>     for D in $(< audit-names.txt)
>     do (cd "$D" && exec sleep 3001)&
>     done
>     sleep 3001
>   done
> 
> Which has the effect of marking the relevant directories as the
> active current directories of each 'sleep' process, and none of
> those directories were "disappeared" from the 'audit' active
> rules list.
> 
> The 'inotify' code has a comment that claims:
> 
> > * inode: Pinned so long as the inode is associated with a watch, from
> > * inotify_add_watch() to the final put_inotify_watch().
> 
> They use 'igrab'/'iput', and 'audit_tree.c' and 'audit_watch.c'
> uses them, so I wonder what is missing.
> 
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit

Re: peculiar disappearance of most audit rules

[Peter Grandi]

[ .... ]

> What's the kernel in question?

Ubuntu 12.04's 3.2 and SteamOS 3.10.

> audit hasn't used "inotify" in a long time.  We now use
> "fsnotify".

Out of laziness I used 'inotify' to mean both; also at one point
I was looking at some 2.6.x sources as there seemed to be
relevant changes in some mailing list.

> but in either case, the inodes aren't supposed to be able to
> be kicked out of core...

But on 3 different system I have they really seem to be evicted,
and with regularity, and this does not happen if the inodes are
kept open.

>From the source I have looked at, the *notify code seems to
attempt to hold on to the inodes that are watched, but perhaps
it has some hidden assumptions that the 'audit' module does not
satisfy.

Re: peculiar disappearance of most audit rules

[Richard Guy Briggs]

On 14/04/27, Peter Grandi wrote:
> > but in either case, the inodes aren't supposed to be able to
> > be kicked out of core...
> 
> But on 3 different system I have they really seem to be evicted,
> and with regularity, and this does not happen if the inodes are
> kept open.
> 
> From the source I have looked at, the *notify code seems to
> attempt to hold on to the inodes that are watched, but perhaps
> it has some hidden assumptions that the 'audit' module does not
> satisfy.

Do you have a reproducer to detect this quickly?

Miklos Szeredi appears to have found the likely cause:
	https://lkml.org/lkml/2014/11/4/246

- RGB

--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545