From mboxrd@z Thu Jan 1 00:00:00 1970 From: Casey Schaufler Subject: Re: Linux audit performance impact Date: Fri, 20 Feb 2015 10:51:35 -0800 Message-ID: <54E78237.8080200@schaufler-ca.com> References: <9DBA79E0CE64AA42B07DEDAAD0F7DB914165C5B7@G4W3222.americas.hpqcorp.net> <20150212182526.GX29998@madcap2.tricolour.ca> <9DBA79E0CE64AA42B07DEDAAD0F7DB914FA921C0@G9W0755.americas.hpqcorp.net> <2039734.h1hbe1jZF6@x2> <9DBA79E0CE64AA42B07DEDAAD0F7DB914FA92535@G9W0755.americas.hpqcorp.net> <20150218211341.GT18752@madcap2.tricolour.ca> <54E77D15.5000601@schaufler-ca.com> <54E77EDD.5030103@ll.mit.edu> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mx1.redhat.com (ext-mx13.extmail.prod.ext.phx2.redhat.com [10.5.110.18]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id t1KJIHxG021732 for ; Fri, 20 Feb 2015 14:18:17 -0500 Received: from smtp102.biz.mail.bf1.yahoo.com (smtp102.biz.mail.bf1.yahoo.com [98.139.221.61]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id t1KJIA7G014839 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Fri, 20 Feb 2015 14:18:14 -0500 In-Reply-To: <54E77EDD.5030103@ll.mit.edu> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linux-audit-bounces@redhat.com Errors-To: linux-audit-bounces@redhat.com To: Ed Christiansen MS , Paul Moore , Richard Guy Briggs Cc: "linux-audit@redhat.com" List-Id: linux-audit@redhat.com On 2/20/2015 10:37 AM, Ed Christiansen MS wrote: > As a guy who administers Irix today I can say the auditing on Irix is > extensive, but I'd hesitate to reference it in this context because > the satd does NOT give you the option to choose success or failure > audits. You get both and it fills your disk fairly quickly. I've > had to disable it during periods of high activity because it will > halt your system (also not configurable) if it runs out of space. So, > maybe it didn't require much in the way of structure, but it left an > awful lot to be desire in the implementation. Yoiks! I was reasonable sure we'd fixed the success/failure choice. Sorry 'bout that. > > On 2/20/2015 1:29 PM, Casey Schaufler wrote: >> On 2/18/2015 1:49 PM, Paul Moore wrote: >>> On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs >>> wrote: >>>> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote: >>>>> I agree that changing the formatting of the records could break >>>>> the existing applications >>>>> that consume them, and I didn't mean changing or eliminating of >>>>> the formatting completely. >>>>> We agree that formatting is required for logging the records(as >>>>> buffers) into the log files. >>>>> We are wondering if these records can be made available as RAW >>>>> records so that the >>>>> analytical programs which are capable of reading them for >>>>> processing can perform better. >>>> There are tools that completely ignore any of the audit userspace >>>> suite >>>> including libaudit, so changing the formatting in the kernel and >>>> deferring to userspace to later do that formatting is not currently an >>>> option. >>> It is if you take a versioned API approach where the kernel defaults >>> to the current behavior and switches, per-socket/connection, at the >>> request of userspace. It's really the only way to have a graceful >>> transition with audit. >>> >>>>> This option of RAW mode for the events can be an additional option >>>>> where, kauditd delivers the audit buffer without formatting. Any >>>>> comments on this? >>>> For a transition period if we were to consider it, it would mean >>>> rewriting *all* places in the kernel that generate audit messages and >>>> provide two paths switched on this RAW mode for each one of them, then >>>> copying all that duplication to userspace libaudit. >>> Your comment is a little vague, so let me mention what I'm currently >>> considering: we convert all of the in-kernel audit users away from >>> generating strings in the context of the caller, instead having them >>> record information in a native/struct/etc. format that would be later >>> used by the kernel audit subsystem to generate the audit records (in >>> whatever format(s) is(are) requested). This actually has advantages >>> beyond the record format work, it moves the issue of record formatting >>> (always a problem) out of the caller and into audit itself which >>> should hopefully prevent future audit abuses (a netlink attribute >>> based record format would likely help further). >> >> The existing audit system is pretty hard on the security modules, too. >> An internal structure that captures the information and formats it later >> makes a whole lot of sense provided the information required to do the >> formatting is available at that later time. It also allows for >> flexibility >> in adding new information to audit records. A new security module could >> add information it considers "security relevant" that other modules >> don't >> without mucking up the audit records from existing modules. >> >> In Irix (The kids on the list can look that up elsewhere :) ) audit >> data was gathered as a collection of audit tokens, each of which >> contained a chuck of information such as the MLS label, or the DAC >> attributes of a process. The tokens were combined to create a complete >> record late in the processing. The scheme didn't require much in the >> way of structure. >> >> I've done several audit systems and would be happy to contribute >> to a revision of the Linux implementation. >> >>> >>>> According to Linus' decree, it would need to remain that way until we >>>> were certain that all tools including ones we don't know about had >>>> switched over. >>> I would imagine a scenario where we introduced the new format in >>> stages: >>> >>> #1 - Move in-kernel audit record string generation completely into >>> kernel/audit*.c. Benefits everyone regardless of the audit format. >>> >>> #2 - Introduce a versioned audit API. The most difficult step for >>> obvious reasons. >>> >>> #3 - Deprecate the old/existing audit record format, make it a Kconfig >>> option that defaults to off and emit a warning when the old formatting >>> is used. This will be a year, and most likely more, after step #2. >>> >>> #4 - Remove the old/existing audit record code. Once again, this >>> would happen a couple of years after step #3. >>> >>> However, nothing is really determined yet, this is just my current >>> thinking. >>> >> >> -- >> Linux-audit mailing list >> Linux-audit@redhat.com >> https://www.redhat.com/mailman/listinfo/linux-audit >> >