* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-23 13:28 UTC (permalink / raw)
To: Paul Moore, Casey Schaufler; +Cc: Richard Guy Briggs, linux-audit@redhat.com
In-Reply-To: <CAHC9VhSjO9pRp9RQ7e0ub+8mosso+acTHVj7geiitxW0pFgVvg@mail.gmail.com>
> -----Original Message-----
> From: linux-audit-bounces@redhat.com [mailto:linux-audit-
> bounces@redhat.com] On Behalf Of Paul Moore
> Sent: Saturday, February 21, 2015 2:52 AM
> To: Casey Schaufler
> Cc: Richard Guy Briggs; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> Yep. However, just so we're clear, what I'm proposing is just a change in the
> kernel API and record format, ultimately the on disk format will be
> dependent on the audit userspace. The good news is that if we can move
> away from this fixed string format it opens the door for different log formats;
> you could stick with the existing goofy strings or switch to any other format
> you like, you just have to write the daemon/tools.
>
> I may end up writing some dummy tools just as part of the kernel
> development process, and I might even maintain them as a simple example
> of an audit userspace. However, my hope is that Steve will update his audit
> userspace to take advantage of the new API when it is ready.
>
>
> My main goal is to try and create a sane API/record-format for the kernel
> that is maintainable over time and feature creep. My secondary goal is to
> push as much processing out of the kernel as possible, both for performance
> and flexibility reasons (see my main goal). A binary record format based
> around netlink attributes is likely the path of least resistance for these goals.
>
> Well, good news, you're in the right place. My patches will be posted here
> and all are welcome, and encouraged, to provide their comments and/or
> patches.
We believe this idea of "handing over the unformatted/binary audit record to audit user space"
gives flexibility to the audit user space to decide on how to handle it and brings
down the overhead that it causes to the system services.
We are also thinking to contribute to this change of linux audit implementation
with the experience of handling auditing on HP-UX.
Regards,
Logeswari.
^ permalink raw reply
* multipart messages & delivery guarantees
From: Hassan Sultan @ 2015-02-23 3:15 UTC (permalink / raw)
To: linux-audit
Hi,
Some events, such as execve or socket-related syscalls generate more than
one message, which I'll separate as the "main" message, and then the 'sub'
messages.
Does the audit system guarantee in any way that user-mode will receive
either no message, or all messages for a given event ?
I'm curious to know if for example I could get an execve syscall message,
but no cwd message, for example in case of low-memory condition.
Thanks,
Hassan
^ permalink raw reply
* [PATCH v2 2/3] kernel/audit: reduce mmap_sem hold for mm->exe_file
From: Davidlohr Bueso @ 2015-02-23 2:20 UTC (permalink / raw)
To: akpm; +Cc: linux-mm, linux-kernel, paul, eparis, linux-audit, dave
In-Reply-To: <1424304641-28965-3-git-send-email-dbueso@suse.de>
The mm->exe_file is currently serialized with mmap_sem (shared)
in order to both safely (1) read the file and (2) audit it via
audit_log_d_path(). Good users will, on the other hand, make use
of the more standard get_mm_exe_file(), requiring only holding
the mmap_sem to read the value, and relying on reference counting
to make sure that the exe file won't dissapear underneath us.
Additionally, upon NULL return of get_mm_exe_file, we also call
audit_log_format(ab, " exe=(null)").
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
changes from v1: rebased on top of 1/1.
kernel/audit.c | 22 ++++++++++++++--------
1 file changed, 14 insertions(+), 8 deletions(-)
diff --git a/kernel/audit.c b/kernel/audit.c
index a71cbfe..b446d54 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -43,6 +43,7 @@
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
+#include <linux/file.h>
#include <linux/init.h>
#include <linux/types.h>
#include <linux/atomic.h>
@@ -1841,15 +1842,20 @@ EXPORT_SYMBOL(audit_log_task_context);
void audit_log_d_path_exe(struct audit_buffer *ab,
struct mm_struct *mm)
{
- if (!mm) {
- audit_log_format(ab, " exe=(null)");
- return;
- }
+ struct file *exe_file;
+
+ if (!mm)
+ goto out_null;
- down_read(&mm->mmap_sem);
- if (mm->exe_file)
- audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
- up_read(&mm->mmap_sem);
+ exe_file = get_mm_exe_file(mm);
+ if (!exe_file)
+ goto out_null;
+
+ audit_log_d_path(ab, " exe=", &exe_file->f_path);
+ fput(exe_file);
+ return;
+out_null:
+ audit_log_format(ab, " exe=(null)");
}
void audit_log_task_info(struct audit_buffer *ab, struct task_struct *tsk)
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* [PATCH v2 1/3] kernel/audit: consolidate handling of mm->exe_file
From: Davidlohr Bueso @ 2015-02-23 2:20 UTC (permalink / raw)
To: akpm; +Cc: linux-mm, linux-kernel, paul, eparis, linux-audit, dave
In-Reply-To: <1424304641-28965-2-git-send-email-dbueso@suse.de>
This patch adds a audit_log_d_path_exe() helper function
to share how we handle auditing of the exe_file's path.
Used by both audit and auditsc. No functionality is changed.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
changes from v1: created normal function for helper.
kernel/audit.c | 23 +++++++++++++++--------
kernel/audit.h | 3 +++
kernel/auditsc.c | 9 +--------
3 files changed, 19 insertions(+), 16 deletions(-)
diff --git a/kernel/audit.c b/kernel/audit.c
index 72ab759..a71cbfe 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -1838,11 +1838,24 @@ error_path:
}
EXPORT_SYMBOL(audit_log_task_context);
+void audit_log_d_path_exe(struct audit_buffer *ab,
+ struct mm_struct *mm)
+{
+ if (!mm) {
+ audit_log_format(ab, " exe=(null)");
+ return;
+ }
+
+ down_read(&mm->mmap_sem);
+ if (mm->exe_file)
+ audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
+ up_read(&mm->mmap_sem);
+}
+
void audit_log_task_info(struct audit_buffer *ab, struct task_struct *tsk)
{
const struct cred *cred;
char comm[sizeof(tsk->comm)];
- struct mm_struct *mm = tsk->mm;
char *tty;
if (!ab)
@@ -1878,13 +1891,7 @@ void audit_log_task_info(struct audit_buffer *ab, struct task_struct *tsk)
audit_log_format(ab, " comm=");
audit_log_untrustedstring(ab, get_task_comm(comm, tsk));
- if (mm) {
- down_read(&mm->mmap_sem);
- if (mm->exe_file)
- audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
- up_read(&mm->mmap_sem);
- } else
- audit_log_format(ab, " exe=(null)");
+ audit_log_d_path_exe(ab, tsk->mm);
audit_log_task_context(ab);
}
EXPORT_SYMBOL(audit_log_task_info);
diff --git a/kernel/audit.h b/kernel/audit.h
index 1caa0d3..d641f9b 100644
--- a/kernel/audit.h
+++ b/kernel/audit.h
@@ -257,6 +257,9 @@ extern struct list_head audit_filter_list[];
extern struct audit_entry *audit_dupe_rule(struct audit_krule *old);
+extern void audit_log_d_path_exe(struct audit_buffer *ab,
+ struct mm_struct *mm);
+
/* audit watch functions */
#ifdef CONFIG_AUDIT_WATCH
extern void audit_put_watch(struct audit_watch *watch);
diff --git a/kernel/auditsc.c b/kernel/auditsc.c
index dc4ae70..84c74d0 100644
--- a/kernel/auditsc.c
+++ b/kernel/auditsc.c
@@ -2361,7 +2361,6 @@ static void audit_log_task(struct audit_buffer *ab)
kuid_t auid, uid;
kgid_t gid;
unsigned int sessionid;
- struct mm_struct *mm = current->mm;
char comm[sizeof(current->comm)];
auid = audit_get_loginuid(current);
@@ -2376,13 +2375,7 @@ static void audit_log_task(struct audit_buffer *ab)
audit_log_task_context(ab);
audit_log_format(ab, " pid=%d comm=", task_pid_nr(current));
audit_log_untrustedstring(ab, get_task_comm(comm, current));
- if (mm) {
- down_read(&mm->mmap_sem);
- if (mm->exe_file)
- audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
- up_read(&mm->mmap_sem);
- } else
- audit_log_format(ab, " exe=(null)");
+ audit_log_d_path_exe(ab, current->mm);
}
/**
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: [PATCH 1/3] kernel/audit: consolidate handling of mm->exe_file
From: Paul Moore @ 2015-02-22 13:14 UTC (permalink / raw)
To: Davidlohr Bueso; +Cc: akpm, linux-mm, linux-kernel, Eric Paris, linux-audit
In-Reply-To: <1424530825.6539.7.camel@stgolabs.net>
On Sat, Feb 21, 2015 at 10:00 AM, Davidlohr Bueso <dave@stgolabs.net> wrote:
> On Sat, 2015-02-21 at 08:45 -0500, Paul Moore wrote:
>> On Fri, Feb 20, 2015 at 8:23 PM, Davidlohr Bueso <dave@stgolabs.net> wrote:
>> > On Wed, 2015-02-18 at 22:23 -0500, Paul Moore wrote:
>> >> I'd prefer if the audit_log_d_path_exe() helper wasn't a static inline.
>> >
>> > What do you have in mind?
>>
>> Pretty much what I said before, audit_log_d_path_exe() as a
>> traditional function and not an inline. Put the function in
>> kernel/audit.c.
>
> well yes I know that, which is why I showed you the code sizes. Now
> again, do you have any reason? This function will only get less bulky in
> the future.
The code size was pretty negligible from my point of view, not enough
to outweigh my preference for a non-inlined version of the function.
Also, I expect this function will be one of the things that gets
shuffled/reworked in the coming months as we make some architectural
changes to audit.
--
paul moore
www.paul-moore.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/3] kernel/audit: consolidate handling of mm->exe_file
From: Davidlohr Bueso @ 2015-02-21 15:00 UTC (permalink / raw)
To: Paul Moore; +Cc: akpm, linux-mm, linux-kernel, Eric Paris, linux-audit
In-Reply-To: <CAHC9VhQxi3YNPFvmfMS6aceC=mi_LcaLD6gqb2zKEb8K_qnZLQ@mail.gmail.com>
On Sat, 2015-02-21 at 08:45 -0500, Paul Moore wrote:
> On Fri, Feb 20, 2015 at 8:23 PM, Davidlohr Bueso <dave@stgolabs.net> wrote:
> > On Wed, 2015-02-18 at 22:23 -0500, Paul Moore wrote:
> >> I'd prefer if the audit_log_d_path_exe() helper wasn't a static inline.
> >
> > What do you have in mind?
>
> Pretty much what I said before, audit_log_d_path_exe() as a
> traditional function and not an inline. Put the function in
> kernel/audit.c.
well yes I know that, which is why I showed you the code sizes. Now
again, do you have any reason? This function will only get less bulky in
the future.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/3] kernel/audit: consolidate handling of mm->exe_file
From: Paul Moore @ 2015-02-21 13:45 UTC (permalink / raw)
To: Davidlohr Bueso; +Cc: akpm, linux-mm, linux-kernel, Eric Paris, linux-audit
In-Reply-To: <1424481838.6539.2.camel@stgolabs.net>
On Fri, Feb 20, 2015 at 8:23 PM, Davidlohr Bueso <dave@stgolabs.net> wrote:
> On Wed, 2015-02-18 at 22:23 -0500, Paul Moore wrote:
>> I'd prefer if the audit_log_d_path_exe() helper wasn't a static inline.
>
> What do you have in mind?
Pretty much what I said before, audit_log_d_path_exe() as a
traditional function and not an inline. Put the function in
kernel/audit.c.
--
paul moore
www.paul-moore.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: [PATCH 1/3] kernel/audit: consolidate handling of mm->exe_file
From: Davidlohr Bueso @ 2015-02-21 1:23 UTC (permalink / raw)
To: Paul Moore; +Cc: akpm, linux-mm, linux-kernel, Eric Paris, linux-audit
In-Reply-To: <CAHC9VhR212FmSEhV_2yryt0=YxTN34ktZ8vveBD3kv4Uhd4WTw@mail.gmail.com>
On Wed, 2015-02-18 at 22:23 -0500, Paul Moore wrote:
> I'd prefer if the audit_log_d_path_exe() helper wasn't a static inline.
What do you have in mind? At least in code size static inlining wins:
text data bss dec hex filename
14423 284 676 15383 3c17 kernel/audit.o
14407 284 676 15367 3c07 kernel/audit.o-thispatch
14474 284 676 15434 3c4a kernel/audit.o-noninline
Thanks,
Davidlohr
^ permalink raw reply
* Re: Linux audit performance impact
From: Paul Moore @ 2015-02-20 21:25 UTC (permalink / raw)
To: Ed Christiansen MS; +Cc: Richard Guy Briggs, linux-audit@redhat.com
In-Reply-To: <54E77EDD.5030103@ll.mit.edu>
On Fri, Feb 20, 2015 at 1:37 PM, Ed Christiansen MS <edwardc@ll.mit.edu> wrote:
> As a guy who administers Irix today I can say the auditing on Irix is
> extensive, but I'd hesitate to reference it in this context because
> the satd does NOT give you the option to choose success or failure
> audits. You get both and it fills your disk fairly quickly. I've
> had to disable it during periods of high activity because it will
> halt your system (also not configurable) if it runs out of space. So,
> maybe it didn't require much in the way of structure, but it left an awful
> lot to be desire in the implementation.
I'm only planning a change in the format, not the content of the audit
records so you'll still have success/fail indicators like you do now.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: Linux audit performance impact
From: Paul Moore @ 2015-02-20 21:22 UTC (permalink / raw)
To: Casey Schaufler; +Cc: Richard Guy Briggs, linux-audit@redhat.com
In-Reply-To: <54E77D15.5000601@schaufler-ca.com>
On Fri, Feb 20, 2015 at 1:29 PM, Casey Schaufler <casey@schaufler-ca.com> wrote:
> The existing audit system is pretty hard on the security modules, too.
Yep.
> An internal structure that captures the information and formats it later
> makes a whole lot of sense provided the information required to do the
> formatting is available at that later time. It also allows for flexibility
> in adding new information to audit records. A new security module could
> add information it considers "security relevant" that other modules don't
> without mucking up the audit records from existing modules.
Yep. However, just so we're clear, what I'm proposing is just a
change in the kernel API and record format, ultimately the on disk
format will be dependent on the audit userspace. The good news is
that if we can move away from this fixed string format it opens the
door for different log formats; you could stick with the existing
goofy strings or switch to any other format you like, you just have to
write the daemon/tools.
I may end up writing some dummy tools just as part of the kernel
development process, and I might even maintain them as a simple
example of an audit userspace. However, my hope is that Steve will
update his audit userspace to take advantage of the new API when it is
ready.
> In Irix (The kids on the list can look that up elsewhere :) ) audit
> data was gathered as a collection of audit tokens, each of which
> contained a chuck of information such as the MLS label, or the DAC
> attributes of a process. The tokens were combined to create a complete
> record late in the processing. The scheme didn't require much in the
> way of structure.
My main goal is to try and create a sane API/record-format for the
kernel that is maintainable over time and feature creep. My secondary
goal is to push as much processing out of the kernel as possible, both
for performance and flexibility reasons (see my main goal). A binary
record format based around netlink attributes is likely the path of
least resistance for these goals.
> I've done several audit systems and would be happy to contribute
> to a revision of the Linux implementation.
Well, good news, you're in the right place. My patches will be posted
here and all are welcome, and encouraged, to provide their comments
and/or patches.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: Linux audit performance impact
From: Casey Schaufler @ 2015-02-20 18:51 UTC (permalink / raw)
To: Ed Christiansen MS, Paul Moore, Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <54E77EDD.5030103@ll.mit.edu>
On 2/20/2015 10:37 AM, Ed Christiansen MS wrote:
> As a guy who administers Irix today I can say the auditing on Irix is
> extensive, but I'd hesitate to reference it in this context because
> the satd does NOT give you the option to choose success or failure
> audits. You get both and it fills your disk fairly quickly. I've
> had to disable it during periods of high activity because it will
> halt your system (also not configurable) if it runs out of space. So,
> maybe it didn't require much in the way of structure, but it left an
> awful lot to be desire in the implementation.
Yoiks! I was reasonable sure we'd fixed the success/failure choice.
Sorry 'bout that.
>
> On 2/20/2015 1:29 PM, Casey Schaufler wrote:
>> On 2/18/2015 1:49 PM, Paul Moore wrote:
>>> On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb@redhat.com>
>>> wrote:
>>>> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
>>>>> I agree that changing the formatting of the records could break
>>>>> the existing applications
>>>>> that consume them, and I didn't mean changing or eliminating of
>>>>> the formatting completely.
>>>>> We agree that formatting is required for logging the records(as
>>>>> buffers) into the log files.
>>>>> We are wondering if these records can be made available as RAW
>>>>> records so that the
>>>>> analytical programs which are capable of reading them for
>>>>> processing can perform better.
>>>> There are tools that completely ignore any of the audit userspace
>>>> suite
>>>> including libaudit, so changing the formatting in the kernel and
>>>> deferring to userspace to later do that formatting is not currently an
>>>> option.
>>> It is if you take a versioned API approach where the kernel defaults
>>> to the current behavior and switches, per-socket/connection, at the
>>> request of userspace. It's really the only way to have a graceful
>>> transition with audit.
>>>
>>>>> This option of RAW mode for the events can be an additional option
>>>>> where, kauditd delivers the audit buffer without formatting. Any
>>>>> comments on this?
>>>> For a transition period if we were to consider it, it would mean
>>>> rewriting *all* places in the kernel that generate audit messages and
>>>> provide two paths switched on this RAW mode for each one of them, then
>>>> copying all that duplication to userspace libaudit.
>>> Your comment is a little vague, so let me mention what I'm currently
>>> considering: we convert all of the in-kernel audit users away from
>>> generating strings in the context of the caller, instead having them
>>> record information in a native/struct/etc. format that would be later
>>> used by the kernel audit subsystem to generate the audit records (in
>>> whatever format(s) is(are) requested). This actually has advantages
>>> beyond the record format work, it moves the issue of record formatting
>>> (always a problem) out of the caller and into audit itself which
>>> should hopefully prevent future audit abuses (a netlink attribute
>>> based record format would likely help further).
>>
>> The existing audit system is pretty hard on the security modules, too.
>> An internal structure that captures the information and formats it later
>> makes a whole lot of sense provided the information required to do the
>> formatting is available at that later time. It also allows for
>> flexibility
>> in adding new information to audit records. A new security module could
>> add information it considers "security relevant" that other modules
>> don't
>> without mucking up the audit records from existing modules.
>>
>> In Irix (The kids on the list can look that up elsewhere :) ) audit
>> data was gathered as a collection of audit tokens, each of which
>> contained a chuck of information such as the MLS label, or the DAC
>> attributes of a process. The tokens were combined to create a complete
>> record late in the processing. The scheme didn't require much in the
>> way of structure.
>>
>> I've done several audit systems and would be happy to contribute
>> to a revision of the Linux implementation.
>>
>>>
>>>> According to Linus' decree, it would need to remain that way until we
>>>> were certain that all tools including ones we don't know about had
>>>> switched over.
>>> I would imagine a scenario where we introduced the new format in
>>> stages:
>>>
>>> #1 - Move in-kernel audit record string generation completely into
>>> kernel/audit*.c. Benefits everyone regardless of the audit format.
>>>
>>> #2 - Introduce a versioned audit API. The most difficult step for
>>> obvious reasons.
>>>
>>> #3 - Deprecate the old/existing audit record format, make it a Kconfig
>>> option that defaults to off and emit a warning when the old formatting
>>> is used. This will be a year, and most likely more, after step #2.
>>>
>>> #4 - Remove the old/existing audit record code. Once again, this
>>> would happen a couple of years after step #3.
>>>
>>> However, nothing is really determined yet, this is just my current
>>> thinking.
>>>
>>
>> --
>> Linux-audit mailing list
>> Linux-audit@redhat.com
>> https://www.redhat.com/mailman/listinfo/linux-audit
>>
>
^ permalink raw reply
* Re: Linux audit performance impact
From: Ed Christiansen MS @ 2015-02-20 18:37 UTC (permalink / raw)
To: Casey Schaufler, Paul Moore, Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <54E77D15.5000601@schaufler-ca.com>
As a guy who administers Irix today I can say the auditing on Irix is
extensive, but I'd hesitate to reference it in this context because
the satd does NOT give you the option to choose success or failure
audits. You get both and it fills your disk fairly quickly. I've
had to disable it during periods of high activity because it will
halt your system (also not configurable) if it runs out of space. So,
maybe it didn't require much in the way of structure, but it left an
awful lot to be desire in the implementation.
On 2/20/2015 1:29 PM, Casey Schaufler wrote:
> On 2/18/2015 1:49 PM, Paul Moore wrote:
>> On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb@redhat.com> wrote:
>>> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
>>>> I agree that changing the formatting of the records could break the existing applications
>>>> that consume them, and I didn't mean changing or eliminating of the formatting completely.
>>>> We agree that formatting is required for logging the records(as buffers) into the log files.
>>>> We are wondering if these records can be made available as RAW records so that the
>>>> analytical programs which are capable of reading them for processing can perform better.
>>> There are tools that completely ignore any of the audit userspace suite
>>> including libaudit, so changing the formatting in the kernel and
>>> deferring to userspace to later do that formatting is not currently an
>>> option.
>> It is if you take a versioned API approach where the kernel defaults
>> to the current behavior and switches, per-socket/connection, at the
>> request of userspace. It's really the only way to have a graceful
>> transition with audit.
>>
>>>> This option of RAW mode for the events can be an additional option
>>>> where, kauditd delivers the audit buffer without formatting. Any
>>>> comments on this?
>>> For a transition period if we were to consider it, it would mean
>>> rewriting *all* places in the kernel that generate audit messages and
>>> provide two paths switched on this RAW mode for each one of them, then
>>> copying all that duplication to userspace libaudit.
>> Your comment is a little vague, so let me mention what I'm currently
>> considering: we convert all of the in-kernel audit users away from
>> generating strings in the context of the caller, instead having them
>> record information in a native/struct/etc. format that would be later
>> used by the kernel audit subsystem to generate the audit records (in
>> whatever format(s) is(are) requested). This actually has advantages
>> beyond the record format work, it moves the issue of record formatting
>> (always a problem) out of the caller and into audit itself which
>> should hopefully prevent future audit abuses (a netlink attribute
>> based record format would likely help further).
>
> The existing audit system is pretty hard on the security modules, too.
> An internal structure that captures the information and formats it later
> makes a whole lot of sense provided the information required to do the
> formatting is available at that later time. It also allows for flexibility
> in adding new information to audit records. A new security module could
> add information it considers "security relevant" that other modules don't
> without mucking up the audit records from existing modules.
>
> In Irix (The kids on the list can look that up elsewhere :) ) audit
> data was gathered as a collection of audit tokens, each of which
> contained a chuck of information such as the MLS label, or the DAC
> attributes of a process. The tokens were combined to create a complete
> record late in the processing. The scheme didn't require much in the
> way of structure.
>
> I've done several audit systems and would be happy to contribute
> to a revision of the Linux implementation.
>
>>
>>> According to Linus' decree, it would need to remain that way until we
>>> were certain that all tools including ones we don't know about had
>>> switched over.
>> I would imagine a scenario where we introduced the new format in stages:
>>
>> #1 - Move in-kernel audit record string generation completely into
>> kernel/audit*.c. Benefits everyone regardless of the audit format.
>>
>> #2 - Introduce a versioned audit API. The most difficult step for
>> obvious reasons.
>>
>> #3 - Deprecate the old/existing audit record format, make it a Kconfig
>> option that defaults to off and emit a warning when the old formatting
>> is used. This will be a year, and most likely more, after step #2.
>>
>> #4 - Remove the old/existing audit record code. Once again, this
>> would happen a couple of years after step #3.
>>
>> However, nothing is really determined yet, this is just my current thinking.
>>
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
>
^ permalink raw reply
* Re: Linux audit performance impact
From: Casey Schaufler @ 2015-02-20 18:29 UTC (permalink / raw)
To: Paul Moore, Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <CAHC9VhRo+oW29NfdvT9o5BiGSjEk=17VdFt9TmusD6xEof_ZYg@mail.gmail.com>
On 2/18/2015 1:49 PM, Paul Moore wrote:
> On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb@redhat.com> wrote:
>> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
>>> I agree that changing the formatting of the records could break the existing applications
>>> that consume them, and I didn't mean changing or eliminating of the formatting completely.
>>> We agree that formatting is required for logging the records(as buffers) into the log files.
>>> We are wondering if these records can be made available as RAW records so that the
>>> analytical programs which are capable of reading them for processing can perform better.
>> There are tools that completely ignore any of the audit userspace suite
>> including libaudit, so changing the formatting in the kernel and
>> deferring to userspace to later do that formatting is not currently an
>> option.
> It is if you take a versioned API approach where the kernel defaults
> to the current behavior and switches, per-socket/connection, at the
> request of userspace. It's really the only way to have a graceful
> transition with audit.
>
>>> This option of RAW mode for the events can be an additional option
>>> where, kauditd delivers the audit buffer without formatting. Any
>>> comments on this?
>> For a transition period if we were to consider it, it would mean
>> rewriting *all* places in the kernel that generate audit messages and
>> provide two paths switched on this RAW mode for each one of them, then
>> copying all that duplication to userspace libaudit.
> Your comment is a little vague, so let me mention what I'm currently
> considering: we convert all of the in-kernel audit users away from
> generating strings in the context of the caller, instead having them
> record information in a native/struct/etc. format that would be later
> used by the kernel audit subsystem to generate the audit records (in
> whatever format(s) is(are) requested). This actually has advantages
> beyond the record format work, it moves the issue of record formatting
> (always a problem) out of the caller and into audit itself which
> should hopefully prevent future audit abuses (a netlink attribute
> based record format would likely help further).
The existing audit system is pretty hard on the security modules, too.
An internal structure that captures the information and formats it later
makes a whole lot of sense provided the information required to do the
formatting is available at that later time. It also allows for flexibility
in adding new information to audit records. A new security module could
add information it considers "security relevant" that other modules don't
without mucking up the audit records from existing modules.
In Irix (The kids on the list can look that up elsewhere :) ) audit
data was gathered as a collection of audit tokens, each of which
contained a chuck of information such as the MLS label, or the DAC
attributes of a process. The tokens were combined to create a complete
record late in the processing. The scheme didn't require much in the
way of structure.
I've done several audit systems and would be happy to contribute
to a revision of the Linux implementation.
>
>> According to Linus' decree, it would need to remain that way until we
>> were certain that all tools including ones we don't know about had
>> switched over.
> I would imagine a scenario where we introduced the new format in stages:
>
> #1 - Move in-kernel audit record string generation completely into
> kernel/audit*.c. Benefits everyone regardless of the audit format.
>
> #2 - Introduce a versioned audit API. The most difficult step for
> obvious reasons.
>
> #3 - Deprecate the old/existing audit record format, make it a Kconfig
> option that defaults to off and emit a warning when the old formatting
> is used. This will be a year, and most likely more, after step #2.
>
> #4 - Remove the old/existing audit record code. Once again, this
> would happen a couple of years after step #3.
>
> However, nothing is really determined yet, this is just my current thinking.
>
^ permalink raw reply
* Re: Linux audit performance impact
From: Paul Moore @ 2015-02-19 3:32 UTC (permalink / raw)
To: Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <20150218223225.GY29998@madcap2.tricolour.ca>
On Wed, Feb 18, 2015 at 5:32 PM, Richard Guy Briggs <rgb@redhat.com> wrote:
> On 15/02/18, Paul Moore wrote:
>> I would imagine a scenario where we introduced the new format in stages:
>>
>> #1 - Move in-kernel audit record string generation completely into
>> kernel/audit*.c. Benefits everyone regardless of the audit format.
>
> Ok.
>
>> #2 - Introduce a versioned audit API. The most difficult step for
>> obvious reasons.
>
> That infrastructure should already be in place. We just converted over
> the version field to a bitfield listing the availability of features.
> An initial call can be made to find out if it is supported, then use the
> feature switching bitfield to enable it. We could alternately make a
> different unicast socket available signalling its availability.
Some of the most basic parts of a versioned API are present, but there
are *big* chunks missing.
>> #3 - Deprecate the old/existing audit record format, make it a Kconfig
>> option that defaults to off and emit a warning when the old formatting
>> is used. This will be a year, and most likely more, after step #2.
>>
>> #4 - Remove the old/existing audit record code. Once again, this
>> would happen a couple of years after step #3.
>
> I suspect in practice stesp #3 and #4 could take a lot longer.
You may be right, I consider the times above as minimums. However,
I'm not completely shutting the door on moving things along sooner; I
don't think we have a ton of users. We'll find out.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: [PATCH 1/3] kernel/audit: consolidate handling of mm->exe_file
From: Paul Moore @ 2015-02-19 3:23 UTC (permalink / raw)
To: Davidlohr Bueso
Cc: akpm, linux-mm, linux-kernel, dave, Eric Paris, linux-audit
In-Reply-To: <1424304641-28965-2-git-send-email-dbueso@suse.de>
On Wed, Feb 18, 2015 at 7:10 PM, Davidlohr Bueso <dbueso@suse.de> wrote:
> From: Davidlohr Bueso <dave@stgolabs.net>
>
> This patch adds a audit_log_d_path_exe() helper function
> to share how we handle auditing of the exe_file's path.
> Used by both audit and auditsc. No functionality is changed.
>
> Cc: Paul Moore <paul@paul-moore.com>
> Cc: Eric Paris <eparis@redhat.com>
> Cc: linux-audit@redhat.com
> Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
> ---
>
> Compile tested only.
>
> kernel/audit.c | 9 +--------
> kernel/audit.h | 14 ++++++++++++++
> kernel/auditsc.c | 9 +--------
> 3 files changed, 16 insertions(+), 16 deletions(-)
I'd prefer if the audit_log_d_path_exe() helper wasn't a static inline.
> --- a/kernel/audit.h
> +++ b/kernel/audit.h
> @@ -257,6 +257,20 @@ extern struct list_head audit_filter_list[];
>
> extern struct audit_entry *audit_dupe_rule(struct audit_krule *old);
>
> +static inline void audit_log_d_path_exe(struct audit_buffer *ab,
> + struct mm_struct *mm)
> +{
> + if (!mm) {
> + audit_log_format(ab, " exe=(null)");
> + return;
> + }
> +
> + down_read(&mm->mmap_sem);
> + if (mm->exe_file)
> + audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
> + up_read(&mm->mmap_sem);
> +}
> +
> /* audit watch functions */
> #ifdef CONFIG_AUDIT_WATCH
> extern void audit_put_watch(struct audit_watch *watch);
--
paul moore
www.paul-moore.com
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* [PATCH 2/3] kernel/audit: robustify handling of mm->exe_file
From: Davidlohr Bueso @ 2015-02-19 0:10 UTC (permalink / raw)
To: akpm
Cc: linux-mm, linux-kernel, dave, paul, eparis, linux-audit,
Davidlohr Bueso
In-Reply-To: <1424304641-28965-1-git-send-email-dbueso@suse.de>
From: Davidlohr Bueso <dave@stgolabs.net>
The mm->exe_file is currently serialized with mmap_sem (shared)
in order to both safely (1) read the file and (2) audit it via
audit_log_d_path(). Good users will, on the other hand, make use
of the more standard get_mm_exe_file(), requiring only holding
the mmap_sem to read the value, and relying on reference counting
to make sure that the exe file won't dissapear underneath us.
This is safe as audit_log_d_path() does not need the mmap_sem --
...and if it did we seriously need to fix that.
Additionally, upon NULL return of get_mm_exe_file, we also call
audit_log_format(ab, " exe=(null)").
Cc: Paul Moore <paul@paul-moore.com>
Cc: Eric Paris <eparis@redhat.com>
Cc: linux-audit@redhat.com
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
Compiled tested only.
kernel/audit.h | 24 +++++++++++++++---------
1 file changed, 15 insertions(+), 9 deletions(-)
diff --git a/kernel/audit.h b/kernel/audit.h
index 510901f..17020f0 100644
--- a/kernel/audit.h
+++ b/kernel/audit.h
@@ -20,6 +20,7 @@
*/
#include <linux/fs.h>
+#include <linux/file.h>
#include <linux/audit.h>
#include <linux/skbuff.h>
#include <uapi/linux/mqueue.h>
@@ -260,15 +261,20 @@ extern struct audit_entry *audit_dupe_rule(struct audit_krule *old);
static inline void audit_log_d_path_exe(struct audit_buffer *ab,
struct mm_struct *mm)
{
- if (!mm) {
- audit_log_format(ab, " exe=(null)");
- return;
- }
-
- down_read(&mm->mmap_sem);
- if (mm->exe_file)
- audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
- up_read(&mm->mmap_sem);
+ struct file *exe_file;
+
+ if (!mm)
+ goto out_null;
+
+ exe_file = get_mm_exe_file(mm);
+ if (!exe_file)
+ goto out_null;
+
+ audit_log_d_path(ab, " exe=", &exe_file->f_path);
+ fput(exe_file);
+ return;
+out_null:
+ audit_log_format(ab, " exe=(null)");
}
/* audit watch functions */
--
2.1.4
^ permalink raw reply related
* [PATCH 1/3] kernel/audit: consolidate handling of mm->exe_file
From: Davidlohr Bueso @ 2015-02-19 0:10 UTC (permalink / raw)
To: akpm
Cc: linux-mm, linux-kernel, dave, paul, eparis, linux-audit,
Davidlohr Bueso
In-Reply-To: <1424304641-28965-1-git-send-email-dbueso@suse.de>
From: Davidlohr Bueso <dave@stgolabs.net>
This patch adds a audit_log_d_path_exe() helper function
to share how we handle auditing of the exe_file's path.
Used by both audit and auditsc. No functionality is changed.
Cc: Paul Moore <paul@paul-moore.com>
Cc: Eric Paris <eparis@redhat.com>
Cc: linux-audit@redhat.com
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
Compile tested only.
kernel/audit.c | 9 +--------
kernel/audit.h | 14 ++++++++++++++
kernel/auditsc.c | 9 +--------
3 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/kernel/audit.c b/kernel/audit.c
index 72ab759..9b49f76 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -1842,7 +1842,6 @@ void audit_log_task_info(struct audit_buffer *ab, struct task_struct *tsk)
{
const struct cred *cred;
char comm[sizeof(tsk->comm)];
- struct mm_struct *mm = tsk->mm;
char *tty;
if (!ab)
@@ -1878,13 +1877,7 @@ void audit_log_task_info(struct audit_buffer *ab, struct task_struct *tsk)
audit_log_format(ab, " comm=");
audit_log_untrustedstring(ab, get_task_comm(comm, tsk));
- if (mm) {
- down_read(&mm->mmap_sem);
- if (mm->exe_file)
- audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
- up_read(&mm->mmap_sem);
- } else
- audit_log_format(ab, " exe=(null)");
+ audit_log_d_path_exe(ab, tsk->mm);
audit_log_task_context(ab);
}
EXPORT_SYMBOL(audit_log_task_info);
diff --git a/kernel/audit.h b/kernel/audit.h
index 1caa0d3..510901f 100644
--- a/kernel/audit.h
+++ b/kernel/audit.h
@@ -257,6 +257,20 @@ extern struct list_head audit_filter_list[];
extern struct audit_entry *audit_dupe_rule(struct audit_krule *old);
+static inline void audit_log_d_path_exe(struct audit_buffer *ab,
+ struct mm_struct *mm)
+{
+ if (!mm) {
+ audit_log_format(ab, " exe=(null)");
+ return;
+ }
+
+ down_read(&mm->mmap_sem);
+ if (mm->exe_file)
+ audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
+ up_read(&mm->mmap_sem);
+}
+
/* audit watch functions */
#ifdef CONFIG_AUDIT_WATCH
extern void audit_put_watch(struct audit_watch *watch);
diff --git a/kernel/auditsc.c b/kernel/auditsc.c
index dc4ae70..84c74d0 100644
--- a/kernel/auditsc.c
+++ b/kernel/auditsc.c
@@ -2361,7 +2361,6 @@ static void audit_log_task(struct audit_buffer *ab)
kuid_t auid, uid;
kgid_t gid;
unsigned int sessionid;
- struct mm_struct *mm = current->mm;
char comm[sizeof(current->comm)];
auid = audit_get_loginuid(current);
@@ -2376,13 +2375,7 @@ static void audit_log_task(struct audit_buffer *ab)
audit_log_task_context(ab);
audit_log_format(ab, " pid=%d comm=", task_pid_nr(current));
audit_log_untrustedstring(ab, get_task_comm(comm, current));
- if (mm) {
- down_read(&mm->mmap_sem);
- if (mm->exe_file)
- audit_log_d_path(ab, " exe=", &mm->exe_file->f_path);
- up_read(&mm->mmap_sem);
- } else
- audit_log_format(ab, " exe=(null)");
+ audit_log_d_path_exe(ab, current->mm);
}
/**
--
2.1.4
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply related
* Re: Linux audit performance impact
From: Richard Guy Briggs @ 2015-02-18 22:32 UTC (permalink / raw)
To: Paul Moore; +Cc: linux-audit@redhat.com
In-Reply-To: <CAHC9VhRo+oW29NfdvT9o5BiGSjEk=17VdFt9TmusD6xEof_ZYg@mail.gmail.com>
On 15/02/18, Paul Moore wrote:
> On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb@redhat.com> wrote:
> > On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
> >> I agree that changing the formatting of the records could break the existing applications
> >> that consume them, and I didn't mean changing or eliminating of the formatting completely.
> >> We agree that formatting is required for logging the records(as buffers) into the log files.
> >> We are wondering if these records can be made available as RAW records so that the
> >> analytical programs which are capable of reading them for processing can perform better.
> >
> > There are tools that completely ignore any of the audit userspace suite
> > including libaudit, so changing the formatting in the kernel and
> > deferring to userspace to later do that formatting is not currently an
> > option.
>
> It is if you take a versioned API approach where the kernel defaults
> to the current behavior and switches, per-socket/connection, at the
> request of userspace. It's really the only way to have a graceful
> transition with audit.
Agreed.
> >> This option of RAW mode for the events can be an additional option
> >> where, kauditd delivers the audit buffer without formatting. Any
> >> comments on this?
> >
> > For a transition period if we were to consider it, it would mean
> > rewriting *all* places in the kernel that generate audit messages and
> > provide two paths switched on this RAW mode for each one of them, then
> > copying all that duplication to userspace libaudit.
>
> Your comment is a little vague, so let me mention what I'm currently
> considering: we convert all of the in-kernel audit users away from
> generating strings in the context of the caller, instead having them
> record information in a native/struct/etc. format that would be later
> used by the kernel audit subsystem to generate the audit records (in
> whatever format(s) is(are) requested). This actually has advantages
> beyond the record format work, it moves the issue of record formatting
> (always a problem) out of the caller and into audit itself which
> should hopefully prevent future audit abuses (a netlink attribute
> based record format would likely help further).
This approach seems good to me.
> > According to Linus' decree, it would need to remain that way until we
> > were certain that all tools including ones we don't know about had
> > switched over.
>
> I would imagine a scenario where we introduced the new format in stages:
>
> #1 - Move in-kernel audit record string generation completely into
> kernel/audit*.c. Benefits everyone regardless of the audit format.
Ok.
> #2 - Introduce a versioned audit API. The most difficult step for
> obvious reasons.
That infrastructure should already be in place. We just converted over
the version field to a bitfield listing the availability of features.
An initial call can be made to find out if it is supported, then use the
feature switching bitfield to enable it. We could alternately make a
different unicast socket available signalling its availability.
> #3 - Deprecate the old/existing audit record format, make it a Kconfig
> option that defaults to off and emit a warning when the old formatting
> is used. This will be a year, and most likely more, after step #2.
>
> #4 - Remove the old/existing audit record code. Once again, this
> would happen a couple of years after step #3.
I suspect in practice stesp #3 and #4 could take a lot longer.
> However, nothing is really determined yet, this is just my current thinking.
>
> paul moore
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
^ permalink raw reply
* Re: Linux audit performance impact
From: Paul Moore @ 2015-02-18 21:49 UTC (permalink / raw)
To: Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <20150218211341.GT18752@madcap2.tricolour.ca>
On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb@redhat.com> wrote:
> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
>> I agree that changing the formatting of the records could break the existing applications
>> that consume them, and I didn't mean changing or eliminating of the formatting completely.
>> We agree that formatting is required for logging the records(as buffers) into the log files.
>> We are wondering if these records can be made available as RAW records so that the
>> analytical programs which are capable of reading them for processing can perform better.
>
> There are tools that completely ignore any of the audit userspace suite
> including libaudit, so changing the formatting in the kernel and
> deferring to userspace to later do that formatting is not currently an
> option.
It is if you take a versioned API approach where the kernel defaults
to the current behavior and switches, per-socket/connection, at the
request of userspace. It's really the only way to have a graceful
transition with audit.
>> This option of RAW mode for the events can be an additional option
>> where, kauditd delivers the audit buffer without formatting. Any
>> comments on this?
>
> For a transition period if we were to consider it, it would mean
> rewriting *all* places in the kernel that generate audit messages and
> provide two paths switched on this RAW mode for each one of them, then
> copying all that duplication to userspace libaudit.
Your comment is a little vague, so let me mention what I'm currently
considering: we convert all of the in-kernel audit users away from
generating strings in the context of the caller, instead having them
record information in a native/struct/etc. format that would be later
used by the kernel audit subsystem to generate the audit records (in
whatever format(s) is(are) requested). This actually has advantages
beyond the record format work, it moves the issue of record formatting
(always a problem) out of the caller and into audit itself which
should hopefully prevent future audit abuses (a netlink attribute
based record format would likely help further).
> According to Linus' decree, it would need to remain that way until we
> were certain that all tools including ones we don't know about had
> switched over.
I would imagine a scenario where we introduced the new format in stages:
#1 - Move in-kernel audit record string generation completely into
kernel/audit*.c. Benefits everyone regardless of the audit format.
#2 - Introduce a versioned audit API. The most difficult step for
obvious reasons.
#3 - Deprecate the old/existing audit record format, make it a Kconfig
option that defaults to off and emit a warning when the old formatting
is used. This will be a year, and most likely more, after step #2.
#4 - Remove the old/existing audit record code. Once again, this
would happen a couple of years after step #3.
However, nothing is really determined yet, this is just my current thinking.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: Linux audit performance impact
From: Satish Chandra Kilaru @ 2015-02-18 21:21 UTC (permalink / raw)
To: Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <20150218211341.GT18752@madcap2.tricolour.ca>
[-- Attachment #1.1: Type: text/plain, Size: 4168 bytes --]
HI
Why/How will the user space tools switch over if the kernel does not
support raw mode?
Isn't it a chicken&egg issue?
--Satish
On Wed, Feb 18, 2015 at 4:13 PM, Richard Guy Briggs <rgb@redhat.com> wrote:
> On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > I agree that changing the formatting of the records could break the
> existing applications
> > that consume them, and I didn't mean changing or eliminating of the
> formatting completely.
> > We agree that formatting is required for logging the records(as buffers)
> into the log files.
> > We are wondering if these records can be made available as RAW records
> so that the
> > analytical programs which are capable of reading them for processing can
> perform better.
>
> There are tools that completely ignore any of the audit userspace suite
> including libaudit, so changing the formatting in the kernel and
> deferring to userspace to later do that formatting is not currently an
> option.
>
> > This option of RAW mode for the events can be an additional option
> > where, kauditd delivers the audit buffer without formatting. Any
> > comments on this?
>
> For a transition period if we were to consider it, it would mean
> rewriting *all* places in the kernel that generate audit messages and
> provide two paths switched on this RAW mode for each one of them, then
> copying all that duplication to userspace libaudit.
> According to Linus' decree, it would need to remain that way until we
> were certain that all tools including ones we don't know about had
> switched over.
>
> > >On Monday, February 16, 2015 11:25:57 AM Viswanath, Logeswari P wrote:
> > >> I configured the system to audit open system call alone instead of all
> > > >the system calls (our loader program executes) and hence I saw the
> > >> massive improvement in performance. My fix is not causing any change
> > > >in the performance. I wrongly communicated that the fix is causing
> > > >performance improvement. Sorry for that.
> > > >
> > >> As per the perf data, the format_decode is the function where most of
> > >> the time is spent i.e. formatting the record in the buffer before
> > > >delivering the data to user space. We need to eliminate formatting
> > > >records to increase the performance. Any idea why we need to format
> > > >the record and whether can we add an option (RAW) to deliver the
> > > >record without formatting to user space?
> >
> > >Introducing any changes to the format of the record can cause all
> analytical programs, both open source and proprietary, to stop working
> correctly. This cannot be changed.
> > >
> > >I think there is room for improvement however. There are times when
> strings are being glued together and a stpcpy works just fine. There are
> times when a numeric hex conversion is being done and %x is very slow. Same
> with %d.
> > >
> > >The other issue is that the audit system's philosophy has not been to
> optimize the formatting of the event, because events _should_ be rare.
> Meaning that if you are getting hundred of events per second, something is
> seriously wrong with the rules.
> > >
> > >It has been optimized to provide as little impact as possible when
> _not_ generating events. Meaning that we want it as fast as possible in
> letting the system operate normally.
> > >
> > >Again, there is room for improvement in both cases of triggering and
> not triggering events. But the format of events can't really change without
> a lot of coordination. I have a test suite here:
> > >
> > >http://people.redhat.com/sgrubb/audit/ausearch-test-0.5.tar.gz
> > >
> > >That can check that events are searchable by the main audit utility. If
> changes cause that to fail, then its a sign you'll break the whole world.
> > >
> > >-Steve
> >
> >
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating
> Systems, Red Hat
> Remote, Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
>
--
Please Donate to www.wikipedia.org
[-- Attachment #1.2: Type: text/html, Size: 5599 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply
* Re: Linux audit performance impact
From: Richard Guy Briggs @ 2015-02-18 21:13 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL); +Cc: linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914FA92535@G9W0755.americas.hpqcorp.net>
On 15/02/17, Viswanath, Logeswari P (MCOU OSTL) wrote:
> I agree that changing the formatting of the records could break the existing applications
> that consume them, and I didn't mean changing or eliminating of the formatting completely.
> We agree that formatting is required for logging the records(as buffers) into the log files.
> We are wondering if these records can be made available as RAW records so that the
> analytical programs which are capable of reading them for processing can perform better.
There are tools that completely ignore any of the audit userspace suite
including libaudit, so changing the formatting in the kernel and
deferring to userspace to later do that formatting is not currently an
option.
> This option of RAW mode for the events can be an additional option
> where, kauditd delivers the audit buffer without formatting. Any
> comments on this?
For a transition period if we were to consider it, it would mean
rewriting *all* places in the kernel that generate audit messages and
provide two paths switched on this RAW mode for each one of them, then
copying all that duplication to userspace libaudit.
According to Linus' decree, it would need to remain that way until we
were certain that all tools including ones we don't know about had
switched over.
> >On Monday, February 16, 2015 11:25:57 AM Viswanath, Logeswari P wrote:
> >> I configured the system to audit open system call alone instead of all
> > >the system calls (our loader program executes) and hence I saw the
> >> massive improvement in performance. My fix is not causing any change
> > >in the performance. I wrongly communicated that the fix is causing
> > >performance improvement. Sorry for that.
> > >
> >> As per the perf data, the format_decode is the function where most of
> >> the time is spent i.e. formatting the record in the buffer before
> > >delivering the data to user space. We need to eliminate formatting
> > >records to increase the performance. Any idea why we need to format
> > >the record and whether can we add an option (RAW) to deliver the
> > >record without formatting to user space?
>
> >Introducing any changes to the format of the record can cause all analytical programs, both open source and proprietary, to stop working correctly. This cannot be changed.
> >
> >I think there is room for improvement however. There are times when strings are being glued together and a stpcpy works just fine. There are times when a numeric hex conversion is being done and %x is very slow. Same with %d.
> >
> >The other issue is that the audit system's philosophy has not been to optimize the formatting of the event, because events _should_ be rare. Meaning that if you are getting hundred of events per second, something is seriously wrong with the rules.
> >
> >It has been optimized to provide as little impact as possible when _not_ generating events. Meaning that we want it as fast as possible in letting the system operate normally.
> >
> >Again, there is room for improvement in both cases of triggering and not triggering events. But the format of events can't really change without a lot of coordination. I have a test suite here:
> >
> >http://people.redhat.com/sgrubb/audit/ausearch-test-0.5.tar.gz
> >
> >That can check that events are searchable by the main audit utility. If changes cause that to fail, then its a sign you'll break the whole world.
> >
> >-Steve
>
>
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
^ permalink raw reply
* Measured load of running auditd with various options?
From: Michael Sierchio @ 2015-02-17 18:06 UTC (permalink / raw)
To: linux-audit
Hi, all. I am wondering if there has been any work done on the
measured impact of running auditd, under various sets of options. I'm
looking for quantitative studies with good analytics, not anecdotes.
Thanks.
- M
^ permalink raw reply
* Re: [PATCH] audit.c : Code clean up
From: Paul Moore @ 2015-02-17 15:44 UTC (permalink / raw)
To: Ameen Ali; +Cc: linux-audit, linux-kernel
In-Reply-To: <1424122426-10991-1-git-send-email-ameenali023@gmail.com>
On Monday, February 16, 2015 11:33:46 PM Ameen Ali wrote:
> Fixed a coding style issue (unnecessary parentheses , unnecessary braces)
>
> Signed-off-by: Ameen-Ali <Ameenali023@gmail.com>
> ---
> kernel/audit.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
Merged into my next-queue branch, I'll push to audit#next once -rc1 is
released.
> diff --git a/kernel/audit.c b/kernel/audit.c
> index 72ab759..0607e12 100644
> --- a/kernel/audit.c
> +++ b/kernel/audit.c
> @@ -663,7 +663,7 @@ static int audit_netlink_ok(struct sk_buff *skb, u16
> msg_type) case AUDIT_MAKE_EQUIV:
> /* Only support auditd and auditctl in initial pid namespace
> * for now. */
> - if ((task_active_pid_ns(current) != &init_pid_ns))
> + if (task_active_pid_ns(current) != &init_pid_ns)
> return -EPERM;
>
> if (!netlink_capable(skb, CAP_AUDIT_CONTROL))
> @@ -1759,7 +1759,7 @@ void audit_log_name(struct audit_context *context,
> struct audit_names *n, } else
> audit_log_format(ab, " name=(null)");
>
> - if (n->ino != (unsigned long)-1) {
> + if (n->ino != (unsigned long)-1)
> audit_log_format(ab, " inode=%lu"
> " dev=%02x:%02x mode=%#ho"
> " ouid=%u ogid=%u rdev=%02x:%02x",
> @@ -1771,7 +1771,6 @@ void audit_log_name(struct audit_context *context,
> struct audit_names *n, from_kgid(&init_user_ns, n->gid),
> MAJOR(n->rdev),
> MINOR(n->rdev));
> - }
> if (n->osid != 0) {
> char *ctx = NULL;
> u32 len;
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: Linux audit performance impact
From: Steve Grubb @ 2015-02-17 13:25 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL); +Cc: linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914FA92535@G9W0755.americas.hpqcorp.net>
On Tuesday, February 17, 2015 01:10:21 PM Viswanath, Logeswari P wrote:
> I agree that changing the formatting of the records could break the existing
> applications that consume them, and I didn't mean changing or eliminating
> of the formatting completely. We agree that formatting is required for
> logging the records(as buffers) into the log files. We are wondering if
> these records can be made available as RAW records so that the analytical
> programs which are capable of reading them for processing can perform
> better.
There are no analytical programs that can consume them. :-) I'd like to see
exactly what the bottleneck was and the correction you made. Again, this is an
optimization for something that should rarely happen. Or if it does, its less
than 10 a second. Additionally, the open use case is about the worst
performing one besides connect or accept because of the large amounts of data
that could be generated. Also, kill can generate 1000's of records in one
syscall.
So, I'd like to see what was optimized to see if you tweaked just this one
syscall and how different it might be for analytical programs.
-Steve
> This option of RAW mode for the events can be an additional option
> where, kauditd delivers the audit buffer without formatting. Any comments
> on this?
>
>
> >On Monday, February 16, 2015 11:25:57 AM Viswanath, Logeswari P wrote:
> >
> >> I configured the system to audit open system call alone instead of all
> >>
> > >the system calls (our loader program executes) and hence I saw the
> > >
> >> massive improvement in performance. My fix is not causing any change
> >>
> > >in the performance. I wrongly communicated that the fix is causing
> > >performance improvement. Sorry for that.
> > >
> > >
> >> As per the perf data, the format_decode is the function where most of
> >> the time is spent i.e. formatting the record in the buffer before
> >>
> > >delivering the data to user space. We need to eliminate formatting
> > >records to increase the performance. Any idea why we need to format
> > >the record and whether can we add an option (RAW) to deliver the
> > >record without formatting to user space?
>
>
>
> >Introducing any changes to the format of the record can cause all
> >analytical programs, both open source and proprietary, to stop working
> >correctly. This cannot be changed.
> >I think there is room for improvement however. There are times when strings
> >are being glued together and a stpcpy works just fine. There are times
> >when a numeric hex conversion is being done and %x is very slow. Same with
> >%d.
> >The other issue is that the audit system's philosophy has not been to
> >optimize the formatting of the event, because events _should_ be rare.
> >Meaning that if you are getting hundred of events per second, something is
> >seriously wrong with the rules.
> >It has been optimized to provide as little impact as possible when _not_
> >generating events. Meaning that we want it as fast as possible in letting
> >the system operate normally.
> >Again, there is room for improvement in both cases of triggering and not
> >triggering events. But the format of events can't really change without a
> >lot of coordination. I have a test suite here:
> >http://people.redhat.com/sgrubb/audit/ausearch-test-0.5.tar.gz
> >
> >That can check that events are searchable by the main audit utility. If
> >changes cause that to fail, then its a sign you'll break the whole world.
> >
> >-Steve
>
>
>
^ permalink raw reply
* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-17 13:10 UTC (permalink / raw)
To: Steve Grubb, linux-audit@redhat.com; +Cc: Richard Guy Briggs
In-Reply-To: <2039734.h1hbe1jZF6@x2>
I agree that changing the formatting of the records could break the existing applications
that consume them, and I didn't mean changing or eliminating of the formatting completely.
We agree that formatting is required for logging the records(as buffers) into the log files.
We are wondering if these records can be made available as RAW records so that the
analytical programs which are capable of reading them for processing can perform better.
This option of RAW mode for the events can be an additional option where, kauditd delivers
the audit buffer without formatting. Any comments on this?
>On Monday, February 16, 2015 11:25:57 AM Viswanath, Logeswari P wrote:
>> I configured the system to audit open system call alone instead of all
> >the system calls (our loader program executes) and hence I saw the
>> massive improvement in performance. My fix is not causing any change
> >in the performance. I wrongly communicated that the fix is causing
> >performance improvement. Sorry for that.
> >
>> As per the perf data, the format_decode is the function where most of
>> the time is spent i.e. formatting the record in the buffer before
> >delivering the data to user space. We need to eliminate formatting
> >records to increase the performance. Any idea why we need to format
> >the record and whether can we add an option (RAW) to deliver the
> >record without formatting to user space?
>Introducing any changes to the format of the record can cause all analytical programs, both open source and proprietary, to stop working correctly. This cannot be changed.
>
>I think there is room for improvement however. There are times when strings are being glued together and a stpcpy works just fine. There are times when a numeric hex conversion is being done and %x is very slow. Same with %d.
>
>The other issue is that the audit system's philosophy has not been to optimize the formatting of the event, because events _should_ be rare. Meaning that if you are getting hundred of events per second, something is seriously wrong with the rules.
>
>It has been optimized to provide as little impact as possible when _not_ generating events. Meaning that we want it as fast as possible in letting the system operate normally.
>
>Again, there is room for improvement in both cases of triggering and not triggering events. But the format of events can't really change without a lot of coordination. I have a test suite here:
>
>http://people.redhat.com/sgrubb/audit/ausearch-test-0.5.tar.gz
>
>That can check that events are searchable by the main audit utility. If changes cause that to fail, then its a sign you'll break the whole world.
>
>-Steve
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox