Re: peculiar disappearance of most audit rules

Steve Grubb
2014-04-21T18:28:44Z

On Monday, April 21, 2014 06:49:24 PM Peter Grandi wrote:
> Hi, I have started using 'auditd', mostly to monitor various directories
> where packages get installed to check for changes in their contents,
> with rules like:
>
> -w /bin -p wa -k pkg-s
> -w /boot -p wa -k pkg-s
> -w /etc -p wa -k pkg-s
> -w /lib -p wa -k pkg-s
> -w /lib32 -p wa -k pkg-s
> -w /lib64 -p wa -k pkg-s
> -w /opt -p wa -k pkg-s
> -w /usr -p wa -k pkg-s
>
> -w /fs/sozan/loc -p wa -k pkg-l
> -w /fs/sozan/loc32-el5 -p wa -k pkg-l
> -w /fs/sozan/loc64-u12 -p wa -k pkg-l
> -w /fs/sozan/com -p wa -k pkg-l
> -w /fs/sozan/com32-el5 -p wa -k pkg-l
> -w /fs/sozan/com64-u12 -p wa -k pkg-l
>
> After setting them, I can verify that for example creating, updating and
> deleting a file in '/boot' or '/opt' gets reported.
>
> Wheat then happens is that even if I set 'auditctl -e 2' some of the
> rules disappear, usually at around the same time as 'cron.daily' scripts
> run, and some more disappear later. This usually seems to relate to
> times where there some significant IO activity ('mlocate' scan, backup),
> but this is a guess.
>
> For example:
>
> time->Thu Apr 17 07:58:44 2014
> type=CONFIG_CHANGE msg=audit(1397717924.255:37148): op="remove rule"
> dir="/boot" key="pkg-s" list=4 res=1 time->Thu Apr 17 07:59:04 2014
> type=CONFIG_CHANGE msg=audit(1397717944.762:37151): op="remove rule"
> dir="/opt" key="pkg-s" list=4 res=1 time->Thu Apr 17 10:01:02 2014
> type=CONFIG_CHANGE msg=audit(1397725262.301:37157): op="remove rule"
> dir="/fs/sozan/loc64-u12" key="pkg-l" list=4 res=1 time->Thu Apr 17
> 10:01:02 2014
> type=CONFIG_CHANGE msg=audit(1397725262.301:37156): op="remove rule"
> dir="/fs/sozan/loc32-el5" key="pkg-l" list=4 res=1
>
> There is no equivalent line in 'dmesg'.
>
> I understand that the 'audit' kernel modules may remove rules if they
> refer to invalid paths, but all the relevant directories do exist, as
> for example '/boot' and '/opt' are the standard usual directories in the
> "root" tree itself:

What happens is that the text path that you put in a watch is a human
convenience. The kernel doesn't understand strings, it understands numbers. It
changes the path into device and inode information. This is, technically, what
your watch is on. If the inode disappears, then the rule is ejected. Rules can
survive across renames but not deletions.

I don't know what is managing your system, but its probably deleting paths.
You might do a ls -i to get the inode number of the directories and then when
you notice the rules being ejected, re-run the ls -i and see if the inodes
have changed.

-Steve

> $ ls -ldn /boot /opt /fs/sozan/loc64-u12 /fs/sozan/loc32-el5
> drwxr-xr-x 3 0 0 4096 Apr 21 07:22 /boot
> drwxrwsr-x 7 1 1 61 Jul 30 2011 /fs/sozan/loc32-el5
> drwxrwsr-x 5 1 1 39 Oct 4 2011 /fs/sozan/loc64-u12
> drwxr-xr-x 7 0 0 4096 Apr 20 14:52 /opt
>
> $ df /boot/. /opt/. /fs/sozan/loc64-u12/. /fs/sozan/loc32-el5/.
> Filesystem 1M-blocks Used Available Use% Mounted on
> /dev/sda3 24815 16853 4106 81% /
> /dev/sda3 24815 16853 4106 81% /
> /dev/sda6 90048 82355 7694 92% /fs/sozan
> /dev/sda6 90048 82355 7694 92% /fs/sozan
>
> This is happening on two similarly configured Ubuntu 12.04 systems with
> both 3.2 and 3.11 Ubuntu "official" kernels. I also have an AppArmor
> configuration which seem to trigger bugs in AppArmor, but all the
> relative profiles are essentially unchanged.
>
> Eventually around almost all of the rules I have set "disappear". For
> example of all these rules:
>
> LIST_RULES: exit,always dir=/fs/sozan/search (0x10) perm=r key=pkg-r
> LIST_RULES: exit,always dir=/fs/sozan/mlocate (0x11) perm=r key=pkg-r
> ....
> LIST_RULES: exit,always dir=/bin (0x4) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/boot (0x5) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/etc (0x4) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/lib (0x4) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/lib32 (0x6) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/lib64 (0x6) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/opt (0x4) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/usr (0x4) perm=wa key=pkg-s
> LIST_RULES: exit,always dir=/fs/sozan/loc (0xd) perm=wa key=pkg-l
> LIST_RULES: exit,always dir=/fs/sozan/loc32-el5 (0x13) perm=wa key=pkg-l
> LIST_RULES: exit,always dir=/fs/sozan/loc64-u12 (0x13) perm=wa key=pkg-l
> LIST_RULES: exit,always dir=/fs/sozan/com (0xd) perm=wa key=pkg-l
> LIST_RULES: exit,always dir=/fs/sozan/com32-el5 (0x13) perm=wa key=pkg-l
> LIST_RULES: exit,always dir=/fs/sozan/com64-u12 (0x13) perm=wa key=pkg-l
>
> Only the first two have not "disappeared" on one of the systems.
>
> This is rather peculiar, please let me know if it is a configuration
> error, an issue, and any fixes or workaround if available (other than
> running 'auditctl -R /etc/audit/audit.rules' every few minutes via CRON).
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit