Re: kauditd hold queue overflow in 4.11

[Steve Grubb <sgrubb@redhat.com>]

On Saturday, September 9, 2017 6:02:02 AM EDT Laurent Bigonville wrote:
> Le 11/07/17 à 00:23, Paul Moore a écrit :
> > On Mon, Jul 10, 2017 at 4:01 PM, Laurent Bigonville <bigon@debian.org> 
wrote:
> >> Le 10/07/17 à 18:00, Paul Moore a écrit :
> >>> On Mon, Jul 10, 2017 at 10:59 AM, Laurent Bigonville <bigon@debian.org>
> >>> 
> >>> wrote:
> >>>> Hi,
> >>>> 
> >>>> With 4.11.6 (that has been uploaded in debian unstable) I get a lot of
> >>>> messages in dmesg like
> >>>> 
> >>>> [100052.120468] audit: audit_lost=66041 audit_rate_limit=0
> >>>> audit_backlog_limit=8192
> >>>> [100052.120470] audit: kauditd hold queue overflow
> >>>> 
> >>>> And it also seems that the messages are not stored in auditd logs
> >>>> anymore.
> >>>> 
> >>>> https://git.kernel.org/linus/264d509637d95f9404e52ced5003ad352e0f6a26
> >>>> seems
> >>>> to be included in this release
> >>>> 
> >>>> An idea?
> >>> 
> >>> 7
> >>> I'm going to assume that your backlog limit is set to a sane value for
> >>> your system's configuration, so that leaves me with two commits that
> >>> may be of interest:
> >>> 
> >>> * 1df30f8264b8 ("audit: fix the RCU locking for the auditd_connection
> >>> structure")
> >>> 
> >>> This was a manual backport of a v4.12 patch to v4.11, looking now, I
> >>> see it should be in +v4.11.5 so that probably isn't your problem ...
> >>> 
> >>> * c81be52a3ac0 ("audit: fix a race condition with the auditd tracking
> >>> code")
> >>> 
> >>> This patch is relatively new and was just sent up to Linus during the
> >>> next merge window; it's a race condition fix so reproducing it can be
> >>> tricky, although it may be easily reproducible on your system at the
> >>> moment (luck you!).  If you aren't in a position to apply the patch,
> >>> the workaround is rather simple: restart auditd.
> >>> 
> >>> If none of the above works, let me know, but I strongly suspect you're
> >>> tripping over the race condition fixed in that last patch.
> >> 
> >> I didn't test the patch yet, but I restarted the auditd daemon 2 times
> >> and
> >> after that the queue has been flushed and I got all the message since
> >> this
> >> noon in the audit logs.
> > 
> > That sounds right; I'm guessing the patch above should be a more permanent
> > fix.
> The patch should be applied in 4.13-rc7 right?
> 
> It seems to fix the main issue (all the audit messages being logged in
> dmesg) but I can still see from time to time the following message:
> 
> [   14.747565] audit: audit_lost=59 audit_rate_limit=0
> audit_backlog_limit=64 [   14.747566] audit: kauditd hold queue overflow

That is a very low backlog_limit. Is this during boot?

-Steve