From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steve Grubb Subject: Re: Handling -ENOBUFS Date: Wed, 5 Nov 2008 13:19:00 -0500 Message-ID: <200811051319.00359.sgrubb@redhat.com> References: <2c03f9590811050830wc551ca6g3a99c467c5e0b7@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <2c03f9590811050830wc551ca6g3a99c467c5e0b7@mail.gmail.com> Content-Disposition: inline List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linux-audit-bounces@redhat.com Errors-To: linux-audit-bounces@redhat.com To: linux-audit@redhat.com List-Id: linux-audit@redhat.com On Wednesday 05 November 2008 11:30:16 Lucas C. Villa Real wrote: > I'm facing a situation where -ENOBUFS is returned from both > audit_send() and audit_get_reply(). The system is under high stress, > with 250k files being created and having creat() and chmod() syscalls > audited. Is this what you really wanted to audit? :) > Looking the code at lib/netlink.c, I saw that audit_send() doesn't > handle -ENOBUFS. Would it be possible to replace the condition from > "while (retval < 0 && errno == EINTR)" to "while (retval < 0 && (errno > == EINTR || errno == ENOBUFS))" to fix the problem when sending > packets from userspace to kernel? Have you tried that? Does it fix the problem or just hang the utility? > My understanding for the problem in audit_get_reply() is that the I/O > buffers are all full and auditd was just not scheduled at the expected > rate, causing these buffers to overflow. Does that make sense? If you go over the backlog limit, you get a syslog message about that unless you have it set to ignore. My guess would be that you have a general network memory pool depletion and is not related to audit specifically. > If it does, do you have a suggestion about the best way to approach this > problem, besides changing auditd's priority? Increase the backlog and increase auditd's priority. I have not played with running auditd with a different scheduler policy than whatever the default is. But you may want to see if one of the other scheduler polices treat audit better. or maybe you want to tune /proc/sys/kernel/sched_granularity_ns. > One interesting thing which I noticed is that 'auditctl -s' doesn't > report that messages were lost, They weren't lost by the audit system so it doesn't know they didn't arrive. > This is happening with an old kernel, 2.6.16.46 + a bunch of patches, > and audit 1.7.4. I cannot completely upgrade it to a new release, but > I can certainly backport audit specific bits if you remember having > fixed something similar since then. Well, that proc tunable is only available for the CFS scheduler. Not sure what you have for older kernels. -Steve