Re: Linux audit performance impact

[Casey Schaufler <casey@schaufler-ca.com>]

On 2/20/2015 10:37 AM, Ed Christiansen MS wrote:
> As a guy who administers Irix today I can say the auditing on Irix is
> extensive, but I'd hesitate to reference it in this context because
> the satd does NOT give you the option to choose success or failure
> audits.  You get both and it fills your disk fairly quickly.  I've
> had to disable it during periods of high activity because it will
> halt your system (also not configurable) if it runs out of space.  So,
> maybe it didn't require much in the way of structure, but it left an
> awful lot to be desire in the implementation.

Yoiks! I was reasonable sure we'd fixed the success/failure choice.
Sorry 'bout that.

>
> On 2/20/2015 1:29 PM, Casey Schaufler wrote:
>> On 2/18/2015 1:49 PM, Paul Moore wrote:
>>> On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb@redhat.com>
>>> wrote:
>>>> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
>>>>> I agree that changing the formatting of the records could break
>>>>> the existing applications
>>>>> that consume them, and I didn't mean changing or eliminating of
>>>>> the formatting completely.
>>>>> We agree that formatting is required for logging the records(as
>>>>> buffers) into the log files.
>>>>> We are wondering if these records can be made available as RAW
>>>>> records so that the
>>>>> analytical programs which are capable of reading them for
>>>>> processing can perform better.
>>>> There are tools that completely ignore any of the audit userspace
>>>> suite
>>>> including libaudit, so changing the formatting in the kernel and
>>>> deferring to userspace to later do that formatting is not currently an
>>>> option.
>>> It is if you take a versioned API approach where the kernel defaults
>>> to the current behavior and switches, per-socket/connection, at the
>>> request of userspace.  It's really the only way to have a graceful
>>> transition with audit.
>>>
>>>>> This option of RAW mode for the events can be an additional option
>>>>> where, kauditd delivers the audit buffer without formatting. Any
>>>>> comments on this?
>>>> For a transition period if we were to consider it, it would mean
>>>> rewriting *all* places in the kernel that generate audit messages and
>>>> provide two paths switched on this RAW mode for each one of them, then
>>>> copying all that duplication to userspace libaudit.
>>> Your comment is a little vague, so let me mention what I'm currently
>>> considering: we convert all of the in-kernel audit users away from
>>> generating strings in the context of the caller, instead having them
>>> record information in a native/struct/etc. format that would be later
>>> used by the kernel audit subsystem to generate the audit records (in
>>> whatever format(s) is(are) requested).  This actually has advantages
>>> beyond the record format work, it moves the issue of record formatting
>>> (always a problem) out of the caller and into audit itself which
>>> should hopefully prevent future audit abuses (a netlink attribute
>>> based record format would likely help further).
>>
>> The existing audit system is pretty hard on the security modules, too.
>> An internal structure that captures the information and formats it later
>> makes a whole lot of sense provided the information required to do the
>> formatting is available at that later time. It also allows for
>> flexibility
>> in adding new information to audit records. A new security module could
>> add information it considers "security relevant" that other modules
>> don't
>> without mucking up the audit records from existing modules.
>>
>> In Irix (The kids on the list can look that up elsewhere :) ) audit
>> data was gathered as a collection of audit tokens, each of which
>> contained a chuck of information such as the MLS label, or the DAC
>> attributes of a process. The tokens were combined to create a complete
>> record late in the processing. The scheme didn't require much in the
>> way of structure.
>>
>> I've done several audit systems and would be happy to contribute
>> to a revision of the Linux implementation.
>>
>>>
>>>> According to Linus' decree, it would need to remain that way until we
>>>> were certain that all tools including ones we don't know about had
>>>> switched over.
>>> I would imagine a scenario where we introduced the new format in
>>> stages:
>>>
>>> #1 - Move in-kernel audit record string generation completely into
>>> kernel/audit*.c.  Benefits everyone regardless of the audit format.
>>>
>>> #2 - Introduce a versioned audit API.  The most difficult step for
>>> obvious reasons.
>>>
>>> #3 - Deprecate the old/existing audit record format, make it a Kconfig
>>> option that defaults to off and emit a warning when the old formatting
>>> is used.  This will be a year, and most likely more, after step #2.
>>>
>>> #4 - Remove the old/existing audit record code.  Once again, this
>>> would happen a couple of years after step #3.
>>>
>>> However, nothing is really determined yet, this is just my current
>>> thinking.
>>>
>>
>> -- 
>> Linux-audit mailing list
>> Linux-audit@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-audit
>>
>