From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andy Ruch Subject: Re: Backlog exceeded when using audisp Date: Thu, 14 Aug 2014 13:12:19 -0700 Message-ID: <1408047139.57007.YahooMailNeo@web120702.mail.ne1.yahoo.com> Reply-To: Andy Ruch Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: Received: from mx1.redhat.com (ext-mx16.extmail.prod.ext.phx2.redhat.com [10.5.110.21]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s7EKCU1R016241 for ; Thu, 14 Aug 2014 16:12:30 -0400 Received: from nm21-vm3.bullet.mail.ne1.yahoo.com (nm21-vm3.bullet.mail.ne1.yahoo.com [98.138.91.151]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s7EKCNpw021949 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 14 Aug 2014 16:12:24 -0400 List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linux-audit-bounces@redhat.com Errors-To: linux-audit-bounces@redhat.com To: Audit ML List-Id: linux-audit@redhat.com > On Wednesday, August 13, 2014 02:37:52 PM Andy Ruch wrote: > > I'm trying to send the audit logs on a secure RHEL 6.5 system to rsyslo= g. > > Rsyslog will then send them to another system for centralized collectio= n. = > > This is fine. = > > > I can't have audisp send them directly because the connectivity is > > unreliable and rsyslog provides on disk queues for reliable delivery. = > > So does auditd. It doesn't lose events unless some limit was exceeded. = > > > I've activated the syslogplugin of audisp to do the transfer. The probl= em is > > getting the logs transferred fast enough. The system is configured to p= anic > > upon error (-f 2), which it does frequently when I do something like up= date > > the SELinux RPM since watching /etc/selinux is required by the STIG. = > > A couple of thoughts here. Perhaps you want to have a policy where when y= ou = > update selinux policy, you suspend auditing and then update and then resu= me. = > Short of that, you'll need to boost the priority and enlarge the queue si= zes. = > The panic will only occur on an in-kernel buffer overflow. User space can= 't do = > that. = > > > I have the audit buffer size configured to 8192 and the audisp queue se= t to > > 120. I'm surprised the 8192 buffer is being overwhelmed. When I look at > > aureport for just the time frame of the action, I get approximately 350 > > events. I know that each event may have multiple entries, but it is > > interesting that the capacity of a buffer over 20 times bigger is being > > exceeded. = > > Well, if the auditspd buffer is full and you have lossless buffering, the= daemon = > waits until there is room in the queue to continue processing and then th= e = > kernel buffer backs up. You have to have settings in user space that allo= w = > auditd to keep the kernel queue as empty as possible. = > > > = > > Can anyone in a similar situation share any insights? Is there a faster= way > > to transfer the logs rather than the audispsyslogplugin? We use to have > > rsyslog monitor the audit.log file but ran into some issues when we sta= rted > > dealing with log file rollover. And it just seems cleaner to send the a= udit > > logs directly. = > > You can also just load rules via auditctl. The kernel defaults to sending = > events to syslog if auditd is not running. = > = > -Steve =A0 Upon further testing, I think there might be something else as the root cau= se. For my testing, I'm adding an selinux user (semanage user -a ...). This= will trigger a load_policy command for SELinux. When everything works fine= , auditd processes roughly 2000 events and audisp handles this with no prob= lems. However, sometimes when I run the command, auditd will receive over 1= 5000 events. As far as I can tell, the extra events are all SELinux error e= vents stating "selinux_audit_rule_match: stale rule". What would cause this= and why does it not happen every time? Is this an audit issue or an SELinu= x issue?