* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-17 13:10 UTC (permalink / raw)
To: Steve Grubb, linux-audit@redhat.com; +Cc: Richard Guy Briggs
In-Reply-To: <2039734.h1hbe1jZF6@x2>
I agree that changing the formatting of the records could break the existing applications
that consume them, and I didn't mean changing or eliminating of the formatting completely.
We agree that formatting is required for logging the records(as buffers) into the log files.
We are wondering if these records can be made available as RAW records so that the
analytical programs which are capable of reading them for processing can perform better.
This option of RAW mode for the events can be an additional option where, kauditd delivers
the audit buffer without formatting. Any comments on this?
>On Monday, February 16, 2015 11:25:57 AM Viswanath, Logeswari P wrote:
>> I configured the system to audit open system call alone instead of all
> >the system calls (our loader program executes) and hence I saw the
>> massive improvement in performance. My fix is not causing any change
> >in the performance. I wrongly communicated that the fix is causing
> >performance improvement. Sorry for that.
> >
>> As per the perf data, the format_decode is the function where most of
>> the time is spent i.e. formatting the record in the buffer before
> >delivering the data to user space. We need to eliminate formatting
> >records to increase the performance. Any idea why we need to format
> >the record and whether can we add an option (RAW) to deliver the
> >record without formatting to user space?
>Introducing any changes to the format of the record can cause all analytical programs, both open source and proprietary, to stop working correctly. This cannot be changed.
>
>I think there is room for improvement however. There are times when strings are being glued together and a stpcpy works just fine. There are times when a numeric hex conversion is being done and %x is very slow. Same with %d.
>
>The other issue is that the audit system's philosophy has not been to optimize the formatting of the event, because events _should_ be rare. Meaning that if you are getting hundred of events per second, something is seriously wrong with the rules.
>
>It has been optimized to provide as little impact as possible when _not_ generating events. Meaning that we want it as fast as possible in letting the system operate normally.
>
>Again, there is room for improvement in both cases of triggering and not triggering events. But the format of events can't really change without a lot of coordination. I have a test suite here:
>
>http://people.redhat.com/sgrubb/audit/ausearch-test-0.5.tar.gz
>
>That can check that events are searchable by the main audit utility. If changes cause that to fail, then its a sign you'll break the whole world.
>
>-Steve
^ permalink raw reply
* [PATCH] audit.c : Code clean up
From: Ameen Ali @ 2015-02-16 21:33 UTC (permalink / raw)
To: paul; +Cc: linux-audit, linux-kernel, Ameen Ali, Ameen-Ali
Fixed a coding style issue (unnecessary parentheses , unnecessary braces)
Signed-off-by: Ameen-Ali <Ameenali023@gmail.com>
---
kernel/audit.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/kernel/audit.c b/kernel/audit.c
index 72ab759..0607e12 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -663,7 +663,7 @@ static int audit_netlink_ok(struct sk_buff *skb, u16 msg_type)
case AUDIT_MAKE_EQUIV:
/* Only support auditd and auditctl in initial pid namespace
* for now. */
- if ((task_active_pid_ns(current) != &init_pid_ns))
+ if (task_active_pid_ns(current) != &init_pid_ns)
return -EPERM;
if (!netlink_capable(skb, CAP_AUDIT_CONTROL))
@@ -1759,7 +1759,7 @@ void audit_log_name(struct audit_context *context, struct audit_names *n,
} else
audit_log_format(ab, " name=(null)");
- if (n->ino != (unsigned long)-1) {
+ if (n->ino != (unsigned long)-1)
audit_log_format(ab, " inode=%lu"
" dev=%02x:%02x mode=%#ho"
" ouid=%u ogid=%u rdev=%02x:%02x",
@@ -1771,7 +1771,6 @@ void audit_log_name(struct audit_context *context, struct audit_names *n,
from_kgid(&init_user_ns, n->gid),
MAJOR(n->rdev),
MINOR(n->rdev));
- }
if (n->osid != 0) {
char *ctx = NULL;
u32 len;
--
2.1.0
^ permalink raw reply related
* Re: audit.c patch
From: Paul Moore @ 2015-02-16 21:11 UTC (permalink / raw)
To: Ameen Ali; +Cc: Eric Paris, linux-audit, linux-kernel
In-Reply-To: <CAGWW_W8DYHuwpq_C26vBeZpn-ew1hpxBvJ2+38HVzsENHT3kog@mail.gmail.com>
On Mon, Feb 16, 2015 at 4:07 PM, Ameen Ali <ameenali023@gmail.com> wrote:
> the following attachment which fix kernel code style issues
Please read Documentation/SubmittingPatches in the kernel sources for
information on how to properly format and submit patches for
consideration. Posting patches inline, and not as an attachment, is
very important.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* audit.c patch
From: Ameen Ali @ 2015-02-16 21:07 UTC (permalink / raw)
To: paul, eparis, linux-audit, linux-kernel
[-- Attachment #1.1: Type: text/plain, Size: 60 bytes --]
the following attachment which fix kernel code style issues
[-- Attachment #1.2: Type: text/html, Size: 92 bytes --]
[-- Attachment #2: 0001-audit.c-Code-clean-up.patch --]
[-- Type: text/x-patch, Size: 1460 bytes --]
From 84ddb5e562db579175079e567a1976444f78f213 Mon Sep 17 00:00:00 2001
From: Ameen Ali <ameenali023@gmail.com>
Date: Mon, 16 Feb 2015 22:00:12 +0200
Subject: [PATCH] audit.c : Code clean up
Fixed a coding style issue (unnecessary parentheses , unnecessary braces)
Signed-off-by: Ameen-Ali <Ameenali023@gmail.com>
---
kernel/audit.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/kernel/audit.c b/kernel/audit.c
index 72ab759..0607e12 100644
--- a/kernel/audit.c
+++ b/kernel/audit.c
@@ -663,7 +663,7 @@ static int audit_netlink_ok(struct sk_buff *skb, u16 msg_type)
case AUDIT_MAKE_EQUIV:
/* Only support auditd and auditctl in initial pid namespace
* for now. */
- if ((task_active_pid_ns(current) != &init_pid_ns))
+ if (task_active_pid_ns(current) != &init_pid_ns)
return -EPERM;
if (!netlink_capable(skb, CAP_AUDIT_CONTROL))
@@ -1759,7 +1759,7 @@ void audit_log_name(struct audit_context *context, struct audit_names *n,
} else
audit_log_format(ab, " name=(null)");
- if (n->ino != (unsigned long)-1) {
+ if (n->ino != (unsigned long)-1)
audit_log_format(ab, " inode=%lu"
" dev=%02x:%02x mode=%#ho"
" ouid=%u ogid=%u rdev=%02x:%02x",
@@ -1771,7 +1771,6 @@ void audit_log_name(struct audit_context *context, struct audit_names *n,
from_kgid(&init_user_ns, n->gid),
MAJOR(n->rdev),
MINOR(n->rdev));
- }
if (n->osid != 0) {
char *ctx = NULL;
u32 len;
--
2.1.0
[-- Attachment #3: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply related
* Re: Linux audit performance impact
From: Paul Moore @ 2015-02-16 17:32 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: Richard Guy Briggs, linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914FA921C0@G9W0755.americas.hpqcorp.net>
On Mon, Feb 16, 2015 at 6:25 AM, Viswanath, Logeswari P (MCOU OSTL)
<logeswari.pv@hp.com> wrote:
> I configured the system to audit open system call alone instead of all the system calls (our loader program executes) and hence I saw the massive improvement in performance.
> My fix is not causing any change in the performance. I wrongly communicated that the fix is causing performance improvement. Sorry for that.
>
> As per the perf data, the format_decode is the function where most of the time is spent i.e. formatting the record in the buffer before delivering the data to user space.
> We need to eliminate formatting records to increase the performance.
> Any idea why we need to format the record and whether can we add an option (RAW) to deliver the record without formatting to user space?
As Steve mentioned, the audit record format is very rigid and poorly
designed, any changes will likely cause significant problems with
userspace.
That said, I'm in the process of evaluating how we can move to a
different format which should alleviate a lot of the problems you
mention in this thread.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: Linux audit performance impact
From: Steve Grubb @ 2015-02-16 12:59 UTC (permalink / raw)
To: linux-audit; +Cc: Richard Guy Briggs
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914FA921C0@G9W0755.americas.hpqcorp.net>
On Monday, February 16, 2015 11:25:57 AM Viswanath, Logeswari P wrote:
> I configured the system to audit open system call alone instead of all the
> system calls (our loader program executes) and hence I saw the massive
> improvement in performance. My fix is not causing any change in the
> performance. I wrongly communicated that the fix is causing performance
> improvement. Sorry for that.
>
> As per the perf data, the format_decode is the function where most of the
> time is spent i.e. formatting the record in the buffer before delivering
> the data to user space. We need to eliminate formatting records to increase
> the performance. Any idea why we need to format the record and whether can
> we add an option (RAW) to deliver the record without formatting to user
> space?
Introducing any changes to the format of the record can cause all analytical
programs, both open source and proprietary, to stop working correctly. This
cannot be changed.
I think there is room for improvement however. There are times when strings
are being glued together and a stpcpy works just fine. There are times when a
numeric hex conversion is being done and %x is very slow. Same with %d.
The other issue is that the audit system's philosophy has not been to optimize
the formatting of the event, because events _should_ be rare. Meaning that if
you are getting hundred of events per second, something is seriously wrong
with the rules.
It has been optimized to provide as little impact as possible when _not_
generating events. Meaning that we want it as fast as possible in letting the
system operate normally.
Again, there is room for improvement in both cases of triggering and not
triggering events. But the format of events can't really change without a lot
of coordination. I have a test suite here:
http://people.redhat.com/sgrubb/audit/ausearch-test-0.5.tar.gz
That can check that events are searchable by the main audit utility. If
changes cause that to fail, then its a sign you'll break the whole world.
-Steve
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Thursday, February 12, 2015 11:55 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: Richard Guy Briggs; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/12, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > Hi all,
> >
> > We did profiling of the kernel (using perf tool) during our performance
> > test and below were the top 4 functions for the overhead.
> >
> > 11.33% loader1 [kernel.kallsyms] [k] format_decode
> >
> > 10.40% loader1 [kernel.kallsyms] [k] memcpy
> >
> > 7.46% loader1 [kernel.kallsyms] [k] number.isra.1
> > 6.99% loader1 [kernel.kallsyms] [k] vsnprintf
> >
> > I was unable to attach the entire profiling data of the kernel because it
> > exceeds the limit of 80KB.>
> > >From the perf data, we believed the overhead is because of invoking
> > >audit_log_format function multiple times.>
> > We changed the code to reduce the number of times this function is called.
> > With this change the performance degradation is 20% now compared to the
> > performance without auditing. Without this change the performance
> > degradation is 200% compared to the performance without auditing.
> Those numbers are not insignificant! I am a bit surprised you were able to
> get that much of an improvement with just this class of change.
> > We can publish the code change done tomorrow.
>
> I'd certainly be interested to see the code.
>
> > Please let me know your feedback on this idea.
> >
> > Regards,
> > Logeswari.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Wednesday, February 11, 2015 10:21 PM
> > To: Viswanath, Logeswari P (MCOU OSTL)
> > Cc: linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > Hi all,
> > >
> > > Please find the below the details of the performance test we ran.
> > > It would be great if we get help to identify the reason behind the
> > > degradation and the ways of improving it.
> > >
> > > Kernel Version:
> > > root > uname -r
> > > 3.13.0-36-generic
> > >
> > > OS Version:
> > > Ubuntu 14.04.1
> > >
> > > No. of CPUs:
> > > root > nproc
> > > 24
> > >
> > > Audit Status:
> > > root > auditctl -s
> > > AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> > > lost=57190353 backlog=0
> > >
> > > Rules Configured:
> > > root > auditctl -l
> > > LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
> > >
> > > Attached is the program used to load the system.
> > >
> > > Results:
> > >
> > > Without enabling audit 12.29
> > > With auditing enabled and no rules configured 12.31
> > > With auditing enabled, 1 rule configured but auditd not running -
> > > kauditd logs audit records to syslog via printk 41.02>
> > This would be more meaningful if you hacked the kernel to drain the queue
> > figuratively to /dev/nul to eliminate the effect of auditd draining it,
> > or syslog covering for a missing auditd. This stat doesn't tell us that
> > much since the I/O act can vary significantly per installation. That one
> > rule you chose is pretty unnaturally abusive and needs to be carefully
> > thought out to avoid self-measurement.>
> > > The degradation is around 200%
> > >
> > > Regards,
> > > Logeswari.
> > >
> > > -----Original Message-----
> > > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > > Sent: Wednesday, February 04, 2015 9:46 PM
> > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> > > Subject: Re: Linux audit performance impact
> > >
> > > On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > The intent is to calculate the performance impact by the auditing
> > > > components such as
> > > >
> > > > 1) impact because of kauditd without auditd - but kauditd writes to
> > > > syslog, so we are unable to determine the impact just because of
> > > > kauditd - It is fine even if the audit record is dropped by kauditd.
> > > > Is there any way to do this?> >
> > > Not yet. That is a mode that has not been useful to anyone yet. You
> > > are welcome to hack a custom kernel to disable klog for doing testing
> > > instrumentation.> >
> > > > 2) impact because of running auditd - log format NOLOG
> > > > 3) impact because of running audispd - small plugin is written which
> > > > will just read the audit records and doesn't processes it.
> > > >
> > > > -----Original Message-----
> > > > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > > > Sent: Tuesday, February 03, 2015 10:33 PM
> > > > To: Satish Chandra Kilaru
> > > > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > > > linux-audit@redhat.com
> > > > Subject: Re: Linux audit performance impact
> > > >
> > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > Thanks for The info. But my question was rhetorical... I meant
> > > > > to say that it would not be much... She is trying to bombard the
> > > > > system with open calls ... So lots and lots of events will be
> > > > > generated and kernel has to write down the events some where or
> > > > > discard them...> > >
> > > > Exactly. It is of little practical use. You have to do I/O at some
> > > > point, either to the same disk or another, or to a network interface
> > > > or serial port, otherwise, just chuck it out. You could do a
> > > > performance measurement on a short burst, then drain the queue, but
> > > > what will that actually tell us?> > >
> > > > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com>
wrote:
> > > > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > > > How many events can kernel accumulate without I/o ?
> > > > > >
> > > > > > The kernel default is 64 *buffers*, but I think Fedora and
> > > > > > RHEL set it to 320. It is now possible to set it to "0" which
> > > > > > means limited only by system resources. See "man auditctl", "-b"
> > > > > > option. An event can be made up of several buffers.
> > > > > >
> > > > > > Of course, how long a system lasts before the queue blows up
> > > > > > depends on your rule set...
> > > > > >
> > > > > > However, at the moment, it will still write out to klog if
> > > > > > auditd isn't running.
> > > > > >
> > > > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > > >
> > > > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > > > > > I don't want to disable auditing (i.e. disable audit
> > > > > > > > record
> > > > > >
> > > > > > collection),
> > > > > >
> > > > > > > > but just do not want the records to delivered to user
> > > > > > > > space since I
> > > > > >
> > > > > > want to
> > > > > >
> > > > > > > > remove the I/O overhead while running the performance test.
> > > > > > > > Is there any option for this?
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > > > <javascript:;>
> > > > > >
> > > > > > <javascript:;>]
> > > > > >
> > > > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > > > linux-audit@redhat.com
> > > > > >
> > > > > > <javascript:;>
> > > > > >
> > > > > > > > <javascript:;>
> > > > > > > > Subject: Re: Linux audit performance impact
> > > > > > > >
> > > > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > > > Please read my question as “Is there any option to
> > > > > > > > > configure kaudit not to log audit records to syslog? when
> > > > > > > > > auditd not running.”> > > > > > >
> > > > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > > > audit=0 in
> > > > > >
> > > > > > its
> > > > > >
> > > > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > > > run since
> > > > > >
> > > > > > boot.
> > > > > >
> > > > > > > > If audit=0 is on the kernel boot line, it will be
> > > > > > > > impossible to run
> > > > > >
> > > > > > auditd.
> > > > > >
> > > > > > > > There is a feature request that is likely coming soon that
> > > > > > > > could be
> > > > > > > > useful:
> > > > > > > >
> > > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > > > subscriber is around, then the kernel shouldn't forward audit
> > > > > > > > data to kmsg"
> > > > > > > >
> > > > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > > > Subject: RE: Linux audit performance impact
> > > > > > > > >
> > > > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > > > records to
> > > > > > > >
> > > > > > > > syslog when auditd is running?
> > > > > > > >
> > > > > > > > > This way we can assess the impact of enabling audit
> > > > > > > > > without involving
> > > > > > > >
> > > > > > > > disk I/o overhead.
> > > > > > > >
> > > > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > > >
> > > > > > <javascript:;> <javascript:;>]
> > > > > >
> > > > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > > > To: Steve Grubb
> > > > > >
> > > > > > > > > Cc: linux-audit@redhat.com <javascript:;>
<javascript:;><mailto:
> > > > > > linux-audit@redhat.com <javascript:;>
> > > > > >
> > > > > > > > <javascript:;>>; Viswanath,
> > > > > > > >
> > > > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > > > Subject: Re: Linux audit performance impact
> > > > > > > > >
> > > > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > > > further events
> > > > > > > >
> > > > > > > > leading spiralling of events...
> > > > > > > >
> > > > > > > > > I brought down my server few times with stupid rules...
> > > > > > > > >
> > > > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > > > <sgrubb@redhat.com
> > > > > >
> > > > > > <javascript:;>
> > > > > >
> > > > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > > >
> > > > > > <javascript:;>>> wrote:
> > > > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish
> > > > > > > > > Chandra Kilaru
> > > > > >
> > > > > > wrote:
> > > > > > > > > > Write your own program to receive audit events
> > > > > > > > > > directly without using auditd...
> > > > > > > > > > That should be faster ....
> > > > > > > > > > Auditd will log the events to disk causing more I/o than u
> > > > > > > > > > need...
> > > > > > > > >
> > > > > > > > > But even that is configurable in many ways. You can
> > > > > > > > > decide if you
> > > > > >
> > > > > > want
> > > > > >
> > > > > > > > > logging to disk or not and what kind of assurance that
> > > > > > > > > it made it to disk and the priority of that audit daemon.
> > > > > > > > > Then you also have all
> > > > > >
> > > > > > the
> > > > > >
> > > > > > > > > normal tuning knobs for disk throughput that you would
> > > > > > > > > use for any disk performance critical system.
> > > > > > > > >
> > > > > > > > > -Steve
> > > > > > > > >
> > > > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > > > (MCOU
> > > > > > > > > > OSTL)
> > > > > >
> > > > > > <
> > > > > >
> > > > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > > > logeswari.pv@hp.com <javascript:;>
> > > > > >
> > > > > > > > <javascript:;>>> wrote:
> > > > > > > > > > > Hi Steve,
> > > > > > > > > > >
> > > > > > > > > > > I am Logeswari working for HP.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > > > Suse linux
> > > > > >
> > > > > > to
> > > > > >
> > > > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > > > host based
> > > > > >
> > > > > > IDS.
> > > > > >
> > > > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > > > system calls auditing is enabled for open/close
> > > > > > > > > > > system call which is a HUGE impact on the system
> > > > > > > > > > > performance.
> > > > > > > > > > > It would
> > > > > >
> > > > > > be
> > > > > >
> > > > > > > > > > > great if anyone can help us answering the following
> > > > > > > > > > > questions.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 1) Is this performance impact expected? If yes,
> > > > > > > > > > > what is the
> > > > > > > >
> > > > > > > > reason
> > > > > > > >
> > > > > > > > > > > behind it and can we fix it?
> > > > > > > > > > >
> > > > > > > > > > > 2) Have anyone done any benchmarking for
> > > > > > > > > > > performance
> > > > > >
> > > > > > impact? If
> > > > > >
> > > > > > > > yes,
> > > > > > > >
> > > > > > > > > > > can you please share the numbers and also the
> > > > > > > > > > > steps/programs used the run the same.
> > > > > > > > > > >
> > > > > > > > > > > 3) Help us validating the performance test we have
> > > > > > > > > > > done in
> > > > > >
> > > > > > our
> > > > > >
> > > > > > > > test
> > > > > > > >
> > > > > > > > > > > setup using the steps mentioned along with the results
> > > > > > > > > > > attached.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > > > close system
> > > > > > > >
> > > > > > > > calls.
> > > > > > > >
> > > > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > > > >
> > > > > > > > > > > We used time command to determine how much time the
> > > > > > > > > > > system took
> > > > > >
> > > > > > to
> > > > > >
> > > > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > > > (results attached
> > > > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > > > With-auditing-RAW)
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > System details:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 1 CPU machine
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > *OS Version*
> > > > > > > > > > >
> > > > > > > > > > > RHEL 6.5
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > *Kernel Version*
> > > > > > > > > > >
> > > > > > > > > > > uname –r
> > > > > > > > > > >
> > > > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Note: auditd was occupying 35% of CPU and was
> > > > > > > > > > > sleeping for most
> > > > > >
> > > > > > of
> > > > > >
> > > > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Thanks & Regards,
> > > > > > > > > > >
> > > > > > > > > > > Logeswari.
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Please Donate to
> > > > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Linux-audit mailing list Linux-audit@redhat.com
> > > > > > > > > <javascript:;> <javascript:;>
> > > > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > > > >
> > > > > > > > - RGB
> > > > > > > >
> > > > > > > > --
> > > > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > > > Canada
> > > > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > > > +1.613.693.0684x3545
> > > > > > >
> > > > > > > --
> > > > > > > Please Donate to www.wikipedia.org
> > > > > >
> > > > > > - RGB
> > > > > >
> > > > > > --
> > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > +1.613.693.0684x3545
> > > > >
> > > > > --
> > > > > Please Donate to www.wikipedia.org
> > > >
> > > > - RGB
> > > >
> > > > --
> > > > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > > > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > > > Ottawa, Canada
> > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > +1.613.693.0684x3545
> > >
> > > - RGB
> > >
> > > --
> > > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > > Ottawa, Canada
> > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > +1.613.693.0684x3545
> > >
> > > #include <stdio.h>
> > > #include <stdlib.h>
> > > #include <sys/stat.h>
> > > #include <fcntl.h>
> > > #include <unistd.h>
> > > #include <errno.h>
> > >
> > > void create_load(int iters);
> > > void cleanup();
> > >
> > > int high_rate = 0;
> > > int num_iters = 100000;
> > > int fd1;
> > > char file1[50];
> > > char file2[50];
> > > char dir1[50];
> > > char symlink1[50];
> > >
> > > /* Purpose: To create system load by invoking system calls used by
> > > templates.> >
> > > *
> > > * Note: The unlink(2) of a file can be an expensive operation (i.e.,
> > > event
> > > * rate goes way down).
> > > */
> > >
> > > main(int argc, char **argv) {
> > >
> > > int num_children=1;
> > > int iters;
> > > int i;
> > > char c;
> > >
> > > while ((c = getopt(argc, argv, "hi:")) != -1) {
> > >
> > > switch (c) {
> > >
> > > case 'h':
> > > /*
> > >
> > > * Desire "high" event rate
> > > */
> > >
> > > high_rate = 1;
> > > argc--;
> > > break;
> > >
> > > case 'i':
> > > /*
> > >
> > > * Desire a specified number of iterations
> > > */
> > >
> > > num_iters = atoi(optarg);
> > > argc--;
> > > break;
> > >
> > > default:
> > > fprintf(stderr,"Unknown option: %c\n",optarg);
> > > exit(1);
> > >
> > > }
> > >
> > > }
> > >
> > >
> > > /*if(argv[optind] != NULL) {
> > >
> > > num_children = atoi(argv[optind]);
> > >
> > > } else {
> > >
> > > num_children = 0;
> > >
> > > }
> > > Register cleanup routine */
> > > fprintf(stderr,"Registering cleanup routine...\n");
> > > if (atexit(cleanup) == -1) {
> > >
> > > fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> > >
> > > errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > >
> > > /* fork child processes, if any requested */
> > > for(i=1; i < num_children; i++) {
> > >
> > > if(fork() == 0) {
> > >
> > > printf("child pid: %d\n",getpid());
> > >
> > > /* Setup file names based on child's pid */
> > > sprintf(file1,"./file1_%d",getpid());
> > > sprintf(file2,"./file2_%d",getpid());
> > > sprintf(dir1,"./dir1_%d",getpid());
> > > sprintf(symlink1,"./file1symlink_%d",getpid());
> > >
> > > /* each child creates load */
> > > iters=0;
> > > if (num_iters == -1) {
> > >
> > > while(1) {
> > >
> > > create_load(iters);
> > > iters++;
> > > if( (iters % 1000) == 0) {
> > >
> > > printf("pid %d iteration %d\n",getpid(),iters);
> > >
> > > }
> > >
> > > }
> > >
> > > } else {
> > >
> > > while(iters < num_iters) {
> > >
> > > create_load(iters);
> > > iters++;
> > > if( (iters % 1000) == 0) {
> > >
> > > printf("pid %d iteration %d\n",getpid(),iters);
> > >
> > > }
> > >
> > > }
> > >
> > > }
> > >
> > > }
> > >
> > > }
> > >
> > > /* Parent creates load also */
> > > printf("parent pid: %d\n",getpid());
> > >
> > > /* Setup file names based on parent's pid */
> > > sprintf(file1,"./file1_%d",getpid());
> > > sprintf(file2,"./file2_%d",getpid());
> > > sprintf(dir1,"./dir1_%d",getpid());
> > > sprintf(symlink1,"./file1symlink_%d",getpid());
> > >
> > > iters=0;
> > > if (num_iters == -1) {
> > >
> > > while(1) {
> > >
> > > create_load(iters);
> > > iters++;
> > > if( (iters % 1000) == 0) {
> > >
> > > printf("pid %d iteration %d\n",getpid(),iters);
> > >
> > > }
> > >
> > > }
> > >
> > > } else {
> > >
> > > while(iters < num_iters) {
> > >
> > > create_load(iters);
> > > iters++;
> > > if( (iters % 1000) == 0) {
> > >
> > > printf("pid %d iteration %d\n",getpid(),iters);
> > >
> > > }
> > >
> > > }
> > >
> > > }
> > >
> > > } /* main */
> > >
> > >
> > > void create_load(int iters) {
> > >
> > > int pid;
> > > char *args[2];
> > > struct stat stat_buf;
> > >
> > > fd1 = creat(file1,0x644);
> > > if (fd1 == -1) {
> > >
> > > fprintf(stderr,"pid %d: creat() returned error for file %s,
> > > errno=%d(%s)\n",> >
> > > getpid(),file1,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (close(fd1) == -1) {
> > >
> > > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > fd1 = open(file1, O_RDWR, 0777);
> > > if (fd1 == -1) {
> > >
> > > fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > > /* Chown this file to root instead of user ids so that we don't
> > > generate a
> > >
> > > * non-owned alert when the file is truncated when invoking creat()
> > > again
> > > * as root on an existing file owned by another user.
> > > */
> > >
> > > if (chown(file1,0,0) == -1) {
> > >
> > > fprintf(stderr,"pid %d: chown(%d,%d) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),0,0,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > > if (fchown(fd1,0,0) == -1) {
> > >
> > > fprintf(stderr,"pid %d: fchown(%d,%d) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),0,0,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > > if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> > >
> > > fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH)
> > > returned error, errno=%d(%s)\n",> >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> > >
> > > fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned
> > > error, errno=%d(%s)\n",> >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > >
> > > if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> > >
> > > fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (ftruncate(fd1,7) == -1) {
> > >
> > > fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (close(fd1) == -1) {
> > >
> > > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > > if (truncate(file1,3) == -1) {
> > >
> > > fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (rename(file1,file2) == -1) {
> > >
> > > fprintf(stderr,"pid %d: rename(%s,%s) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),file1,file2,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (rename(file2,file1) == -1) {
> > >
> > > fprintf(stderr,"pid %d: rename(%s,%s) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),file2,file1,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (link(file1,file2) == -1) {
> > >
> > > fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> > >
> > > getpid(),file1,file2,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (symlink(file1,symlink1) == -1) {
> > >
> > > fprintf(stderr,"pid %d: symlink(%s,%s) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),file1,symlink1,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (lchown(symlink1,0,0) == -1) {
> > >
> > > fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),symlink1,0,0,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > > if (lstat(symlink1,&stat_buf) == -1) {
> > >
> > > fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> > >
> > > getpid(),symlink1,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (stat(file1,&stat_buf) == -1) {
> > >
> > > fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> > >
> > > getpid(),file1,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (unlink(file1) == -1) {
> > >
> > > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > >
> > > getpid(),file1,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (unlink(file2) == -1) {
> > >
> > > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > >
> > > getpid(),file2,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (unlink(symlink1) == -1) {
> > >
> > > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > >
> > > getpid(),symlink1,errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> > >
> > > fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > > if (rmdir(dir1) == -1) {
> > >
> > > fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > > /* Fork every 10000 iterations to not use up process resources too
> > > quickly */ if ( (iters % 10000) == 0) {
> > >
> > > pid = fork();
> > > if(pid == 0) {
> > >
> > > fprintf(stderr,"child pid %d: fork!\n",getpid());
> > > // child
> > > args[0] = "/bin/ls";
> > > args[1] = NULL;
> > > close(1);
> > > close(2);
> > > execve(args[0], args, NULL);
> > > fprintf(stderr,"pid %d: execve(%s) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),args[0],errno,strerror(errno));
> > >
> > > _exit(1);
> > >
> > > } else if (pid < 0) {
> > >
> > > fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > } else {
> > >
> > > fprintf(stderr,"parent pid %d, child pid: %d:
> > > fork!\n",getpid(),pid);
> > >
> > > }
> > >
> > > pid = vfork();
> > > if(pid == 0) {
> > >
> > > args[0] = "/bin/pwd";
> > > args[1] = NULL;
> > > close(1);
> > > close(2);
> > > execv(args[0], args);
> > > fprintf(stderr,"pid %d: execve(%s) returned error,
> > > errno=%d(%s)\n",
> > >
> > > getpid(),args[0],errno,strerror(errno));
> > >
> > > _exit(1);
> > >
> > > } else if (pid < 0) {
> > >
> > > fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> > >
> > > getpid(),errno,strerror(errno));
> > >
> > > exit(1);
> > >
> > > }
> > >
> > > }
> > >
> > > /* Make sure everything is cleaned up and deleted before returning */
> > > cleanup();
> > >
> > > } /* create_load() */
> > >
> > > void cleanup() {
> > >
> > > close(fd1);
> > > unlink(file1);
> > > unlink(file2);
> > > unlink(symlink1);
> > > unlink(dir1);
> > > return;
> > >
> > > }
> > >
> > > --
> > > Linux-audit mailing list
> > > Linux-audit@redhat.com
> > > https://www.redhat.com/mailman/listinfo/linux-audit
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
> >
> > --
> > Linux-audit mailing list
> > Linux-audit@redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-audit
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems,
> Red Hat Remote, Ottawa, Canada Voice: +1.647.777.2635, Internal: (81)
> 32635, Alt: +1.613.693.0684x3545
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-16 11:25 UTC (permalink / raw)
To: Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <20150212182526.GX29998@madcap2.tricolour.ca>
I configured the system to audit open system call alone instead of all the system calls (our loader program executes) and hence I saw the massive improvement in performance.
My fix is not causing any change in the performance. I wrongly communicated that the fix is causing performance improvement. Sorry for that.
As per the perf data, the format_decode is the function where most of the time is spent i.e. formatting the record in the buffer before delivering the data to user space.
We need to eliminate formatting records to increase the performance.
Any idea why we need to format the record and whether can we add an option (RAW) to deliver the record without formatting to user space?
-----Original Message-----
From: Richard Guy Briggs [mailto:rgb@redhat.com]
Sent: Thursday, February 12, 2015 11:55 PM
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: Richard Guy Briggs; linux-audit@redhat.com
Subject: Re: Linux audit performance impact
On 15/02/12, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Hi all,
>
> We did profiling of the kernel (using perf tool) during our performance test and below were the top 4 functions for the overhead.
>
> 11.33% loader1 [kernel.kallsyms] [k] format_decode
> 10.40% loader1 [kernel.kallsyms] [k] memcpy
> 7.46% loader1 [kernel.kallsyms] [k] number.isra.1
> 6.99% loader1 [kernel.kallsyms] [k] vsnprintf
>
> I was unable to attach the entire profiling data of the kernel because it exceeds the limit of 80KB.
>
> >From the perf data, we believed the overhead is because of invoking audit_log_format function multiple times.
> We changed the code to reduce the number of times this function is called.
> With this change the performance degradation is 20% now compared to the performance without auditing.
> Without this change the performance degradation is 200% compared to the performance without auditing.
Those numbers are not insignificant! I am a bit surprised you were able to get that much of an improvement with just this class of change.
> We can publish the code change done tomorrow.
I'd certainly be interested to see the code.
> Please let me know your feedback on this idea.
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 11, 2015 10:21 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > Hi all,
> >
> > Please find the below the details of the performance test we ran.
> > It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
> >
> > Kernel Version:
> > root > uname -r
> > 3.13.0-36-generic
> >
> > OS Version:
> > Ubuntu 14.04.1
> >
> > No. of CPUs:
> > root > nproc
> > 24
> >
> > Audit Status:
> > root > auditctl -s
> > AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> > lost=57190353 backlog=0
> >
> > Rules Configured:
> > root > auditctl -l
> > LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
> >
> > Attached is the program used to load the system.
> >
> > Results:
> >
> > Without enabling audit 12.29
> > With auditing enabled and no rules configured 12.31
> > With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
>
> This would be more meaningful if you hacked the kernel to drain the queue figuratively to /dev/nul to eliminate the effect of auditd draining it, or syslog covering for a missing auditd. This stat doesn't tell us that much since the I/O act can vary significantly per installation. That one rule you chose is pretty unnaturally abusive and needs to be carefully thought out to avoid self-measurement.
>
> > The degradation is around 200%
> >
> > Regards,
> > Logeswari.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Wednesday, February 04, 2015 9:46 PM
> > To: Viswanath, Logeswari P (MCOU OSTL)
> > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > The intent is to calculate the performance impact by the auditing
> > > components such as
> > >
> > > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
> >
> > Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
> >
> > > 2) impact because of running auditd - log format NOLOG
> > > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> > >
> > > -----Original Message-----
> > > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > > Sent: Tuesday, February 03, 2015 10:33 PM
> > > To: Satish Chandra Kilaru
> > > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > > linux-audit@redhat.com
> > > Subject: Re: Linux audit performance impact
> > >
> > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > Thanks for The info. But my question was rhetorical... I meant
> > > > to say that it would not be much... She is trying to bombard the
> > > > system with open calls ... So lots and lots of events will be
> > > > generated and kernel has to write down the events some where or discard them...
> > >
> > > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> > >
> > > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > > >
> > > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > > How many events can kernel accumulate without I/o ?
> > > > >
> > > > > The kernel default is 64 *buffers*, but I think Fedora and
> > > > > RHEL set it to 320. It is now possible to set it to "0" which
> > > > > means limited only by system resources. See "man auditctl", "-b"
> > > > > option. An event can be made up of several buffers.
> > > > >
> > > > > Of course, how long a system lasts before the queue blows up
> > > > > depends on your rule set...
> > > > >
> > > > > However, at the moment, it will still write out to klog if
> > > > > auditd isn't running.
> > > > >
> > > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > > >
> > > > > > > I don't want to disable auditing (i.e. disable audit
> > > > > > > record
> > > > > collection),
> > > > > > > but just do not want the records to delivered to user
> > > > > > > space since I
> > > > > want to
> > > > > > > remove the I/O overhead while running the performance test.
> > > > > > > Is there any option for this?
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > > <javascript:;>
> > > > > <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > > linux-audit@redhat.com
> > > > > <javascript:;>
> > > > > > > <javascript:;>
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > > Please read my question as “Is there any option to
> > > > > > > > configure kaudit not to log audit records to syslog? when auditd not running.”
> > > > > > >
> > > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > > audit=0 in
> > > > > its
> > > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > > run since
> > > > > boot.
> > > > > > > If audit=0 is on the kernel boot line, it will be
> > > > > > > impossible to run
> > > > > auditd.
> > > > > > >
> > > > > > > There is a feature request that is likely coming soon that
> > > > > > > could be
> > > > > > > useful:
> > > > > > >
> > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > > >
> > > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > > Subject: RE: Linux audit performance impact
> > > > > > > >
> > > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > > records to
> > > > > > > syslog when auditd is running?
> > > > > > > > This way we can assess the impact of enabling audit
> > > > > > > > without involving
> > > > > > > disk I/o overhead.
> > > > > > > >
> > > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > > <javascript:;> <javascript:;>]
> > > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > > To: Steve Grubb
> > > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > > linux-audit@redhat.com <javascript:;>
> > > > > > > <javascript:;>>; Viswanath,
> > > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > > Subject: Re: Linux audit performance impact
> > > > > > > >
> > > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > > further events
> > > > > > > leading spiralling of events...
> > > > > > > > I brought down my server few times with stupid rules...
> > > > > > > >
> > > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > > <sgrubb@redhat.com
> > > > > <javascript:;>
> > > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > > <javascript:;>>> wrote:
> > > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish
> > > > > > > > Chandra Kilaru
> > > > > wrote:
> > > > > > > > > Write your own program to receive audit events
> > > > > > > > > directly without using auditd...
> > > > > > > > > That should be faster ....
> > > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > > >
> > > > > > > > But even that is configurable in many ways. You can
> > > > > > > > decide if you
> > > > > want
> > > > > > > > logging to disk or not and what kind of assurance that
> > > > > > > > it made it to disk and the priority of that audit daemon.
> > > > > > > > Then you also have all
> > > > > the
> > > > > > > > normal tuning knobs for disk throughput that you would
> > > > > > > > use for any disk performance critical system.
> > > > > > > >
> > > > > > > > -Steve
> > > > > > > >
> > > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > > (MCOU
> > > > > > > > > OSTL)
> > > > > <
> > > > > > > > >
> > > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > > logeswari.pv@hp.com <javascript:;>
> > > > > > > <javascript:;>>> wrote:
> > > > > > > > > > Hi Steve,
> > > > > > > > > >
> > > > > > > > > > I am Logeswari working for HP.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > > Suse linux
> > > > > to
> > > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > > host based
> > > > > IDS.
> > > > > > > > > >
> > > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > > system calls auditing is enabled for open/close
> > > > > > > > > > system call which is a HUGE impact on the system performance.
> > > > > > > > > > It would
> > > > > be
> > > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > > reason
> > > > > > > > > > behind it and can we fix it?
> > > > > > > > > >
> > > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > > impact? If
> > > > > > > yes,
> > > > > > > > > > can you please share the numbers and also the
> > > > > > > > > > steps/programs used the run the same.
> > > > > > > > > >
> > > > > > > > > > 3) Help us validating the performance test we have done in
> > > > > our
> > > > > > > test
> > > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > > close system
> > > > > > > calls.
> > > > > > > > > >
> > > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > > >
> > > > > > > > > > We used time command to determine how much time the
> > > > > > > > > > system took
> > > > > to
> > > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > > (results attached
> > > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > > With-auditing-RAW)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > System details:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1 CPU machine
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > *OS Version*
> > > > > > > > > >
> > > > > > > > > > RHEL 6.5
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > *Kernel Version*
> > > > > > > > > >
> > > > > > > > > > uname –r
> > > > > > > > > >
> > > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Note: auditd was occupying 35% of CPU and was
> > > > > > > > > > sleeping for most
> > > > > of
> > > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thanks & Regards,
> > > > > > > > > >
> > > > > > > > > > Logeswari.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Please Donate to
> > > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > > >
> > > > > > > > --
> > > > > > > > Linux-audit mailing list Linux-audit@redhat.com
> > > > > > > > <javascript:;> <javascript:;>
> > > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > > >
> > > > > > >
> > > > > > > - RGB
> > > > > > >
> > > > > > > --
> > > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > > Canada
> > > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > > +1.613.693.0684x3545
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Please Donate to www.wikipedia.org
> > > > >
> > > > > - RGB
> > > > >
> > > > > --
> > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > +1.613.693.0684x3545
> > > > >
> > > >
> > > >
> > > > --
> > > > Please Donate to www.wikipedia.org
> > >
> > > - RGB
> > >
> > > --
> > > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > > Ottawa, Canada
> > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > +1.613.693.0684x3545
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <sys/stat.h>
> > #include <fcntl.h>
> > #include <unistd.h>
> > #include <errno.h>
> >
> > void create_load(int iters);
> > void cleanup();
> >
> > int high_rate = 0;
> > int num_iters = 100000;
> > int fd1;
> > char file1[50];
> > char file2[50];
> > char dir1[50];
> > char symlink1[50];
> >
> > /* Purpose: To create system load by invoking system calls used by templates.
> > *
> > * Note: The unlink(2) of a file can be an expensive operation (i.e., event
> > * rate goes way down).
> > */
> >
> > main(int argc, char **argv) {
> >
> > int num_children=1;
> > int iters;
> > int i;
> > char c;
> >
> > while ((c = getopt(argc, argv, "hi:")) != -1) {
> > switch (c) {
> > case 'h':
> > /*
> > * Desire "high" event rate
> > */
> > high_rate = 1;
> > argc--;
> > break;
> > case 'i':
> > /*
> > * Desire a specified number of iterations
> > */
> > num_iters = atoi(optarg);
> > argc--;
> > break;
> > default:
> > fprintf(stderr,"Unknown option: %c\n",optarg);
> > exit(1);
> > }
> > }
> >
> >
> > /*if(argv[optind] != NULL) {
> > num_children = atoi(argv[optind]);
> > } else {
> > num_children = 0;
> > }
> > Register cleanup routine */
> > fprintf(stderr,"Registering cleanup routine...\n");
> > if (atexit(cleanup) == -1) {
> > fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> > errno,strerror(errno));
> > exit(1);
> > }
> >
> >
> > /* fork child processes, if any requested */
> > for(i=1; i < num_children; i++) {
> > if(fork() == 0) {
> >
> > printf("child pid: %d\n",getpid());
> >
> > /* Setup file names based on child's pid */
> > sprintf(file1,"./file1_%d",getpid());
> > sprintf(file2,"./file2_%d",getpid());
> > sprintf(dir1,"./dir1_%d",getpid());
> > sprintf(symlink1,"./file1symlink_%d",getpid());
> >
> > /* each child creates load */
> > iters=0;
> > if (num_iters == -1) {
> > while(1) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > } else {
> > while(iters < num_iters) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > }
> > }
> > }
> >
> > /* Parent creates load also */
> > printf("parent pid: %d\n",getpid());
> >
> > /* Setup file names based on parent's pid */
> > sprintf(file1,"./file1_%d",getpid());
> > sprintf(file2,"./file2_%d",getpid());
> > sprintf(dir1,"./dir1_%d",getpid());
> > sprintf(symlink1,"./file1symlink_%d",getpid());
> >
> > iters=0;
> > if (num_iters == -1) {
> > while(1) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > } else {
> > while(iters < num_iters) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > }
> >
> > } /* main */
> >
> >
> > void create_load(int iters) {
> >
> > int pid;
> > char *args[2];
> > struct stat stat_buf;
> >
> > fd1 = creat(file1,0x644);
> > if (fd1 == -1) {
> > fprintf(stderr,"pid %d: creat() returned error for file %s, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (close(fd1) == -1) {
> > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > fd1 = open(file1, O_RDWR, 0777);
> > if (fd1 == -1) {
> > fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > /* Chown this file to root instead of user ids so that we don't generate a
> > * non-owned alert when the file is truncated when invoking creat() again
> > * as root on an existing file owned by another user.
> > */
> > if (chown(file1,0,0) == -1) {
> > fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> > getpid(),0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (fchown(fd1,0,0) == -1) {
> > fprintf(stderr,"pid %d: fchown(%d,%d) returned error, errno=%d(%s)\n",
> > getpid(),0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> > fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> > fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> >
> > if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> > fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (ftruncate(fd1,7) == -1) {
> > fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (close(fd1) == -1) {
> > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (truncate(file1,3) == -1) {
> > fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (rename(file1,file2) == -1) {
> > fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (rename(file2,file1) == -1) {
> > fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file2,file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (link(file1,file2) == -1) {
> > fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (symlink(file1,symlink1) == -1) {
> > fprintf(stderr,"pid %d: symlink(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (lchown(symlink1,0,0) == -1) {
> > fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (lstat(symlink1,&stat_buf) == -1) {
> > fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (stat(file1,&stat_buf) == -1) {
> > fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(file1) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(file2) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(symlink1) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> > fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (rmdir(dir1) == -1) {
> > fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > /* Fork every 10000 iterations to not use up process resources too quickly */
> > if ( (iters % 10000) == 0) {
> > pid = fork();
> > if(pid == 0) {
> > fprintf(stderr,"child pid %d: fork!\n",getpid());
> > // child
> > args[0] = "/bin/ls";
> > args[1] = NULL;
> > close(1);
> > close(2);
> > execve(args[0], args, NULL);
> > fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> > getpid(),args[0],errno,strerror(errno));
> > _exit(1);
> > } else if (pid < 0) {
> > fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > } else {
> > fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
> > }
> >
> > pid = vfork();
> > if(pid == 0) {
> > args[0] = "/bin/pwd";
> > args[1] = NULL;
> > close(1);
> > close(2);
> > execv(args[0], args);
> > fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> > getpid(),args[0],errno,strerror(errno));
> > _exit(1);
> > } else if (pid < 0) {
> > fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > }
> >
> > /* Make sure everything is cleaned up and deleted before returning */
> > cleanup();
> >
> > } /* create_load() */
> >
> > void cleanup() {
> > close(fd1);
> > unlink(file1);
> > unlink(file2);
> > unlink(symlink1);
> > unlink(dir1);
> > return;
> > }
>
> > --
> > Linux-audit mailing list
> > Linux-audit@redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-audit
>
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> +1.613.693.0684x3545
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* Re: Linux audit performance impact
From: Satish Chandra Kilaru @ 2015-02-13 14:15 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: Richard Guy Briggs, linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914165C592@G4W3222.americas.hpqcorp.net>
[-- Attachment #1.1: Type: text/plain, Size: 26809 bytes --]
Excellent!!!
On Thu, Feb 12, 2015 at 9:58 AM, Viswanath, Logeswari P (MCOU OSTL) <
logeswari.pv@hp.com> wrote:
> Hi all,
>
> We did profiling of the kernel during our performance test and below were
> the top 4 functions for the overhead.
>
> 11.33% loader1 [kernel.kallsyms] [k] format_decode
> 10.40% loader1 [kernel.kallsyms] [k] memcpy
> 7.46% loader1 [kernel.kallsyms] [k] number.isra.1
> 6.99% loader1 [kernel.kallsyms] [k] vsnprintf
>
> Please find attached the complete profiling data of the kernel using perf
> tool.
>
> From the perf data, we believed the overhead is because of invoking
> audit_log_format function multiple times.
> We changed the code to reduce the number of times this function is called.
> With this change the performance degradation is 20% now compared to the
> performance without auditing.
> Without this change the performance degradation is 200% compared to the
> performance without auditing.
>
> We can publish the code change done tomorrow.
>
> Please let me know your feedback on this idea.
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 11, 2015 10:21 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > Hi all,
> >
> > Please find the below the details of the performance test we ran.
> > It would be great if we get help to identify the reason behind the
> degradation and the ways of improving it.
> >
> > Kernel Version:
> > root > uname -r
> > 3.13.0-36-generic
> >
> > OS Version:
> > Ubuntu 14.04.1
> >
> > No. of CPUs:
> > root > nproc
> > 24
> >
> > Audit Status:
> > root > auditctl -s
> > AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> > lost=57190353 backlog=0
> >
> > Rules Configured:
> > root > auditctl -l
> > LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
> >
> > Attached is the program used to load the system.
> >
> > Results:
> >
> > Without enabling audit 12.29
> > With auditing enabled and no rules configured 12.31
> > With auditing enabled, 1 rule configured but auditd not running -
> kauditd logs audit records to syslog via printk 41.02
>
> This would be more meaningful if you hacked the kernel to drain the queue
> figuratively to /dev/nul to eliminate the effect of auditd draining it, or
> syslog covering for a missing auditd. This stat doesn't tell us that much
> since the I/O act can vary significantly per installation. That one rule
> you chose is pretty unnaturally abusive and needs to be carefully thought
> out to avoid self-measurement.
>
> > The degradation is around 200%
> >
> > Regards,
> > Logeswari.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Wednesday, February 04, 2015 9:46 PM
> > To: Viswanath, Logeswari P (MCOU OSTL)
> > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > The intent is to calculate the performance impact by the auditing
> > > components such as
> > >
> > > 1) impact because of kauditd without auditd - but kauditd writes to
> syslog, so we are unable to determine the impact just because of kauditd -
> It is fine even if the audit record is dropped by kauditd. Is there any way
> to do this?
> >
> > Not yet. That is a mode that has not been useful to anyone yet. You
> are welcome to hack a custom kernel to disable klog for doing testing
> instrumentation.
> >
> > > 2) impact because of running auditd - log format NOLOG
> > > 3) impact because of running audispd - small plugin is written which
> will just read the audit records and doesn't processes it.
> > >
> > > -----Original Message-----
> > > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > > Sent: Tuesday, February 03, 2015 10:33 PM
> > > To: Satish Chandra Kilaru
> > > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > > linux-audit@redhat.com
> > > Subject: Re: Linux audit performance impact
> > >
> > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > Thanks for The info. But my question was rhetorical... I meant to
> > > > say that it would not be much... She is trying to bombard the
> > > > system with open calls ... So lots and lots of events will be
> > > > generated and kernel has to write down the events some where or
> discard them...
> > >
> > > Exactly. It is of little practical use. You have to do I/O at some
> point, either to the same disk or another, or to a network interface or
> serial port, otherwise, just chuck it out. You could do a performance
> measurement on a short burst, then drain the queue, but what will that
> actually tell us?
> > >
> > > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com>
> wrote:
> > > >
> > > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > > How many events can kernel accumulate without I/o ?
> > > > >
> > > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > > set it to 320. It is now possible to set it to "0" which means
> > > > > limited only by system resources. See "man auditctl", "-b"
> > > > > option. An event can be made up of several buffers.
> > > > >
> > > > > Of course, how long a system lasts before the queue blows up
> > > > > depends on your rule set...
> > > > >
> > > > > However, at the moment, it will still write out to klog if
> > > > > auditd isn't running.
> > > > >
> > > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > > >
> > > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > > collection),
> > > > > > > but just do not want the records to delivered to user space
> > > > > > > since I
> > > > > want to
> > > > > > > remove the I/O overhead while running the performance test.
> > > > > > > Is there any option for this?
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > > <javascript:;>
> > > > > <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > > linux-audit@redhat.com
> > > > > <javascript:;>
> > > > > > > <javascript:;>
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > > Please read my question as “Is there any option to
> > > > > > > > configure kaudit not to log audit records to syslog? when
> auditd not running.”
> > > > > > >
> > > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > > audit=0 in
> > > > > its
> > > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > > run since
> > > > > boot.
> > > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > > to run
> > > > > auditd.
> > > > > > >
> > > > > > > There is a feature request that is likely coming soon that
> > > > > > > could be
> > > > > > > useful:
> > > > > > >
> > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > > subscriber is around, then the kernel shouldn't forward audit
> data to kmsg"
> > > > > > >
> > > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > > Subject: RE: Linux audit performance impact
> > > > > > > >
> > > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > > records to
> > > > > > > syslog when auditd is running?
> > > > > > > > This way we can assess the impact of enabling audit
> > > > > > > > without involving
> > > > > > > disk I/o overhead.
> > > > > > > >
> > > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > > <javascript:;> <javascript:;>]
> > > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > > To: Steve Grubb
> > > > > > > > Cc: linux-audit@redhat.com <javascript:;>
> <javascript:;><mailto:
> > > > > linux-audit@redhat.com <javascript:;>
> > > > > > > <javascript:;>>; Viswanath,
> > > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > > Subject: Re: Linux audit performance impact
> > > > > > > >
> > > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > > further events
> > > > > > > leading spiralling of events...
> > > > > > > > I brought down my server few times with stupid rules...
> > > > > > > >
> > > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > > <sgrubb@redhat.com
> > > > > <javascript:;>
> > > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > > <javascript:;>>> wrote:
> > > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > > Kilaru
> > > > > wrote:
> > > > > > > > > Write your own program to receive audit events directly
> > > > > > > > > without using auditd...
> > > > > > > > > That should be faster ....
> > > > > > > > > Auditd will log the events to disk causing more I/o than u
> need...
> > > > > > > >
> > > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > > if you
> > > > > want
> > > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > > made it to disk and the priority of that audit daemon.
> > > > > > > > Then you also have all
> > > > > the
> > > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > > for any disk performance critical system.
> > > > > > > >
> > > > > > > > -Steve
> > > > > > > >
> > > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > > (MCOU
> > > > > > > > > OSTL)
> > > > > <
> > > > > > > > >
> > > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > > logeswari.pv@hp.com <javascript:;>
> > > > > > > <javascript:;>>> wrote:
> > > > > > > > > > Hi Steve,
> > > > > > > > > >
> > > > > > > > > > I am Logeswari working for HP.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > > Suse linux
> > > > > to
> > > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > > host based
> > > > > IDS.
> > > > > > > > > >
> > > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > > system calls auditing is enabled for open/close system
> > > > > > > > > > call which is a HUGE impact on the system performance.
> > > > > > > > > > It would
> > > > > be
> > > > > > > > > > great if anyone can help us answering the following
> questions.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1) Is this performance impact expected? If yes,
> what is the
> > > > > > > reason
> > > > > > > > > > behind it and can we fix it?
> > > > > > > > > >
> > > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > > impact? If
> > > > > > > yes,
> > > > > > > > > > can you please share the numbers and also the
> > > > > > > > > > steps/programs used the run the same.
> > > > > > > > > >
> > > > > > > > > > 3) Help us validating the performance test we have
> done in
> > > > > our
> > > > > > > test
> > > > > > > > > > setup using the steps mentioned along with the results
> attached.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > > close system
> > > > > > > calls.
> > > > > > > > > >
> > > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > > >
> > > > > > > > > > We used time command to determine how much time the
> > > > > > > > > > system took
> > > > > to
> > > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > > (results attached
> > > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > > With-auditing-RAW)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > System details:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1 CPU machine
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > *OS Version*
> > > > > > > > > >
> > > > > > > > > > RHEL 6.5
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > *Kernel Version*
> > > > > > > > > >
> > > > > > > > > > uname –r
> > > > > > > > > >
> > > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > > for most
> > > > > of
> > > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thanks & Regards,
> > > > > > > > > >
> > > > > > > > > > Logeswari.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Please Donate to
> > > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > > >
> > > > > > > > --
> > > > > > > > Linux-audit mailing list
> > > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > > >
> > > > > > >
> > > > > > > - RGB
> > > > > > >
> > > > > > > --
> > > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > > Canada
> > > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > > +1.613.693.0684x3545
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Please Donate to www.wikipedia.org
> > > > >
> > > > > - RGB
> > > > >
> > > > > --
> > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > +1.613.693.0684x3545
> > > > >
> > > >
> > > >
> > > > --
> > > > Please Donate to www.wikipedia.org
> > >
> > > - RGB
> > >
> > > --
> > > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > > Ottawa, Canada
> > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > +1.613.693.0684x3545
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <sys/stat.h>
> > #include <fcntl.h>
> > #include <unistd.h>
> > #include <errno.h>
> >
> > void create_load(int iters);
> > void cleanup();
> >
> > int high_rate = 0;
> > int num_iters = 100000;
> > int fd1;
> > char file1[50];
> > char file2[50];
> > char dir1[50];
> > char symlink1[50];
> >
> > /* Purpose: To create system load by invoking system calls used by
> templates.
> > *
> > * Note: The unlink(2) of a file can be an expensive operation (i.e.,
> event
> > * rate goes way down).
> > */
> >
> > main(int argc, char **argv) {
> >
> > int num_children=1;
> > int iters;
> > int i;
> > char c;
> >
> > while ((c = getopt(argc, argv, "hi:")) != -1) {
> > switch (c) {
> > case 'h':
> > /*
> > * Desire "high" event rate
> > */
> > high_rate = 1;
> > argc--;
> > break;
> > case 'i':
> > /*
> > * Desire a specified number of iterations
> > */
> > num_iters = atoi(optarg);
> > argc--;
> > break;
> > default:
> > fprintf(stderr,"Unknown option: %c\n",optarg);
> > exit(1);
> > }
> > }
> >
> >
> > /*if(argv[optind] != NULL) {
> > num_children = atoi(argv[optind]);
> > } else {
> > num_children = 0;
> > }
> > Register cleanup routine */
> > fprintf(stderr,"Registering cleanup routine...\n");
> > if (atexit(cleanup) == -1) {
> > fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> > errno,strerror(errno));
> > exit(1);
> > }
> >
> >
> > /* fork child processes, if any requested */
> > for(i=1; i < num_children; i++) {
> > if(fork() == 0) {
> >
> > printf("child pid: %d\n",getpid());
> >
> > /* Setup file names based on child's pid */
> > sprintf(file1,"./file1_%d",getpid());
> > sprintf(file2,"./file2_%d",getpid());
> > sprintf(dir1,"./dir1_%d",getpid());
> > sprintf(symlink1,"./file1symlink_%d",getpid());
> >
> > /* each child creates load */
> > iters=0;
> > if (num_iters == -1) {
> > while(1) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > } else {
> > while(iters < num_iters) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > }
> > }
> > }
> >
> > /* Parent creates load also */
> > printf("parent pid: %d\n",getpid());
> >
> > /* Setup file names based on parent's pid */
> > sprintf(file1,"./file1_%d",getpid());
> > sprintf(file2,"./file2_%d",getpid());
> > sprintf(dir1,"./dir1_%d",getpid());
> > sprintf(symlink1,"./file1symlink_%d",getpid());
> >
> > iters=0;
> > if (num_iters == -1) {
> > while(1) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > } else {
> > while(iters < num_iters) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > }
> >
> > } /* main */
> >
> >
> > void create_load(int iters) {
> >
> > int pid;
> > char *args[2];
> > struct stat stat_buf;
> >
> > fd1 = creat(file1,0x644);
> > if (fd1 == -1) {
> > fprintf(stderr,"pid %d: creat() returned error for file %s,
> errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (close(fd1) == -1) {
> > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > fd1 = open(file1, O_RDWR, 0777);
> > if (fd1 == -1) {
> > fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > /* Chown this file to root instead of user ids so that we don't
> generate a
> > * non-owned alert when the file is truncated when invoking creat()
> again
> > * as root on an existing file owned by another user.
> > */
> > if (chown(file1,0,0) == -1) {
> > fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> > getpid(),0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (fchown(fd1,0,0) == -1) {
> > fprintf(stderr,"pid %d: fchown(%d,%d) returned error,
> errno=%d(%s)\n",
> > getpid(),0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> > fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH)
> returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> > fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned
> error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> >
> > if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> > fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (ftruncate(fd1,7) == -1) {
> > fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (close(fd1) == -1) {
> > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (truncate(file1,3) == -1) {
> > fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (rename(file1,file2) == -1) {
> > fprintf(stderr,"pid %d: rename(%s,%s) returned error,
> errno=%d(%s)\n",
> > getpid(),file1,file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (rename(file2,file1) == -1) {
> > fprintf(stderr,"pid %d: rename(%s,%s) returned error,
> errno=%d(%s)\n",
> > getpid(),file2,file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (link(file1,file2) == -1) {
> > fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (symlink(file1,symlink1) == -1) {
> > fprintf(stderr,"pid %d: symlink(%s,%s) returned error,
> errno=%d(%s)\n",
> > getpid(),file1,symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (lchown(symlink1,0,0) == -1) {
> > fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error,
> errno=%d(%s)\n",
> > getpid(),symlink1,0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (lstat(symlink1,&stat_buf) == -1) {
> > fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (stat(file1,&stat_buf) == -1) {
> > fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(file1) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(file2) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(symlink1) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> > fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (rmdir(dir1) == -1) {
> > fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > /* Fork every 10000 iterations to not use up process resources too
> quickly */
> > if ( (iters % 10000) == 0) {
> > pid = fork();
> > if(pid == 0) {
> > fprintf(stderr,"child pid %d: fork!\n",getpid());
> > // child
> > args[0] = "/bin/ls";
> > args[1] = NULL;
> > close(1);
> > close(2);
> > execve(args[0], args, NULL);
> > fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> > getpid(),args[0],errno,strerror(errno));
> > _exit(1);
> > } else if (pid < 0) {
> > fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > } else {
> > fprintf(stderr,"parent pid %d, child pid: %d:
> fork!\n",getpid(),pid);
> > }
> >
> > pid = vfork();
> > if(pid == 0) {
> > args[0] = "/bin/pwd";
> > args[1] = NULL;
> > close(1);
> > close(2);
> > execv(args[0], args);
> > fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> > getpid(),args[0],errno,strerror(errno));
> > _exit(1);
> > } else if (pid < 0) {
> > fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > }
> >
> > /* Make sure everything is cleaned up and deleted before returning */
> > cleanup();
> >
> > } /* create_load() */
> >
> > void cleanup() {
> > close(fd1);
> > unlink(file1);
> > unlink(file2);
> > unlink(symlink1);
> > unlink(dir1);
> > return;
> > }
>
> > --
> > Linux-audit mailing list
> > Linux-audit@redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-audit
>
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating
> Systems, Red Hat Remote, Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
>
--
Please Donate to www.wikipedia.org
[-- Attachment #1.2: Type: text/html, Size: 39314 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply
* RE: Audit: audisp-remote plugin: why is OpenLDAP a dependency ??
From: Keith A. Glass @ 2015-02-12 22:22 UTC (permalink / raw)
To: 'Steve Grubb'; +Cc: linux-audit
In-Reply-To: <158203247.f4u9RDS02R@x2>
Actually, I was trying to do the re-spin, but it's been overcome by events,
just shipping the output of auditd to /var/log/messages, and letting Splunk
sort it all out. . .
-----Original Message-----
From: Steve Grubb [mailto:sgrubb@redhat.com]
Sent: Thursday, February 12, 2015 2:42 PM
To: salgak@comcast.net
Cc: linux-audit@redhat.com
Subject: Re: Audit: audisp-remote plugin: why is OpenLDAP a dependency ??
On Tuesday, February 10, 2015 07:01:27 PM salgak@comcast.net wrote:
> So, respin a new RPM using rpmrebuild, dropping the zos plugins ?
>
> i.e. the appropriate section of the SPEC would now look like this:
I'll see about making a configure switch to not build it, too.
-Steve
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2013.0.3495 / Virus Database: 4257/9101 - Release Date: 02/12/15
^ permalink raw reply
* Re: Audit: audisp-remote plugin: why is OpenLDAP a dependency ??
From: Steve Grubb @ 2015-02-12 19:42 UTC (permalink / raw)
To: salgak; +Cc: linux-audit
In-Reply-To: <285615815.3363270.1423594887279.JavaMail.zimbra@comcast.net>
On Tuesday, February 10, 2015 07:01:27 PM salgak@comcast.net wrote:
> So, respin a new RPM using rpmrebuild, dropping the zos plugins ?
>
> i.e. the appropriate section of the SPEC would now look like this:
I'll see about making a configure switch to not build it, too.
-Steve
^ permalink raw reply
* Re: Linux audit performance impact
From: Richard Guy Briggs @ 2015-02-12 18:25 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: Richard Guy Briggs, linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914165C5B7@G4W3222.americas.hpqcorp.net>
On 15/02/12, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Hi all,
>
> We did profiling of the kernel (using perf tool) during our performance test and below were the top 4 functions for the overhead.
>
> 11.33% loader1 [kernel.kallsyms] [k] format_decode
> 10.40% loader1 [kernel.kallsyms] [k] memcpy
> 7.46% loader1 [kernel.kallsyms] [k] number.isra.1
> 6.99% loader1 [kernel.kallsyms] [k] vsnprintf
>
> I was unable to attach the entire profiling data of the kernel because it exceeds the limit of 80KB.
>
> >From the perf data, we believed the overhead is because of invoking audit_log_format function multiple times.
> We changed the code to reduce the number of times this function is called.
> With this change the performance degradation is 20% now compared to the performance without auditing.
> Without this change the performance degradation is 200% compared to the performance without auditing.
Those numbers are not insignificant! I am a bit surprised you were able
to get that much of an improvement with just this class of change.
> We can publish the code change done tomorrow.
I'd certainly be interested to see the code.
> Please let me know your feedback on this idea.
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 11, 2015 10:21 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > Hi all,
> >
> > Please find the below the details of the performance test we ran.
> > It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
> >
> > Kernel Version:
> > root > uname -r
> > 3.13.0-36-generic
> >
> > OS Version:
> > Ubuntu 14.04.1
> >
> > No. of CPUs:
> > root > nproc
> > 24
> >
> > Audit Status:
> > root > auditctl -s
> > AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> > lost=57190353 backlog=0
> >
> > Rules Configured:
> > root > auditctl -l
> > LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
> >
> > Attached is the program used to load the system.
> >
> > Results:
> >
> > Without enabling audit 12.29
> > With auditing enabled and no rules configured 12.31
> > With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
>
> This would be more meaningful if you hacked the kernel to drain the queue figuratively to /dev/nul to eliminate the effect of auditd draining it, or syslog covering for a missing auditd. This stat doesn't tell us that much since the I/O act can vary significantly per installation. That one rule you chose is pretty unnaturally abusive and needs to be carefully thought out to avoid self-measurement.
>
> > The degradation is around 200%
> >
> > Regards,
> > Logeswari.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Wednesday, February 04, 2015 9:46 PM
> > To: Viswanath, Logeswari P (MCOU OSTL)
> > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > The intent is to calculate the performance impact by the auditing
> > > components such as
> > >
> > > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
> >
> > Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
> >
> > > 2) impact because of running auditd - log format NOLOG
> > > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> > >
> > > -----Original Message-----
> > > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > > Sent: Tuesday, February 03, 2015 10:33 PM
> > > To: Satish Chandra Kilaru
> > > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > > linux-audit@redhat.com
> > > Subject: Re: Linux audit performance impact
> > >
> > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > Thanks for The info. But my question was rhetorical... I meant to
> > > > say that it would not be much... She is trying to bombard the
> > > > system with open calls ... So lots and lots of events will be
> > > > generated and kernel has to write down the events some where or discard them...
> > >
> > > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> > >
> > > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > > >
> > > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > > How many events can kernel accumulate without I/o ?
> > > > >
> > > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > > set it to 320. It is now possible to set it to "0" which means
> > > > > limited only by system resources. See "man auditctl", "-b"
> > > > > option. An event can be made up of several buffers.
> > > > >
> > > > > Of course, how long a system lasts before the queue blows up
> > > > > depends on your rule set...
> > > > >
> > > > > However, at the moment, it will still write out to klog if
> > > > > auditd isn't running.
> > > > >
> > > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > > >
> > > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > > collection),
> > > > > > > but just do not want the records to delivered to user space
> > > > > > > since I
> > > > > want to
> > > > > > > remove the I/O overhead while running the performance test.
> > > > > > > Is there any option for this?
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > > <javascript:;>
> > > > > <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > > linux-audit@redhat.com
> > > > > <javascript:;>
> > > > > > > <javascript:;>
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > > Please read my question as “Is there any option to
> > > > > > > > configure kaudit not to log audit records to syslog? when auditd not running.”
> > > > > > >
> > > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > > audit=0 in
> > > > > its
> > > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > > run since
> > > > > boot.
> > > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > > to run
> > > > > auditd.
> > > > > > >
> > > > > > > There is a feature request that is likely coming soon that
> > > > > > > could be
> > > > > > > useful:
> > > > > > >
> > > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > > >
> > > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > > Subject: RE: Linux audit performance impact
> > > > > > > >
> > > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > > records to
> > > > > > > syslog when auditd is running?
> > > > > > > > This way we can assess the impact of enabling audit
> > > > > > > > without involving
> > > > > > > disk I/o overhead.
> > > > > > > >
> > > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > > <javascript:;> <javascript:;>]
> > > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > > To: Steve Grubb
> > > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > > linux-audit@redhat.com <javascript:;>
> > > > > > > <javascript:;>>; Viswanath,
> > > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > > Subject: Re: Linux audit performance impact
> > > > > > > >
> > > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > > further events
> > > > > > > leading spiralling of events...
> > > > > > > > I brought down my server few times with stupid rules...
> > > > > > > >
> > > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > > <sgrubb@redhat.com
> > > > > <javascript:;>
> > > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > > <javascript:;>>> wrote:
> > > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > > Kilaru
> > > > > wrote:
> > > > > > > > > Write your own program to receive audit events directly
> > > > > > > > > without using auditd...
> > > > > > > > > That should be faster ....
> > > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > > >
> > > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > > if you
> > > > > want
> > > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > > made it to disk and the priority of that audit daemon.
> > > > > > > > Then you also have all
> > > > > the
> > > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > > for any disk performance critical system.
> > > > > > > >
> > > > > > > > -Steve
> > > > > > > >
> > > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > > (MCOU
> > > > > > > > > OSTL)
> > > > > <
> > > > > > > > >
> > > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > > logeswari.pv@hp.com <javascript:;>
> > > > > > > <javascript:;>>> wrote:
> > > > > > > > > > Hi Steve,
> > > > > > > > > >
> > > > > > > > > > I am Logeswari working for HP.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > > Suse linux
> > > > > to
> > > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > > host based
> > > > > IDS.
> > > > > > > > > >
> > > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > > system calls auditing is enabled for open/close system
> > > > > > > > > > call which is a HUGE impact on the system performance.
> > > > > > > > > > It would
> > > > > be
> > > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > > reason
> > > > > > > > > > behind it and can we fix it?
> > > > > > > > > >
> > > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > > impact? If
> > > > > > > yes,
> > > > > > > > > > can you please share the numbers and also the
> > > > > > > > > > steps/programs used the run the same.
> > > > > > > > > >
> > > > > > > > > > 3) Help us validating the performance test we have done in
> > > > > our
> > > > > > > test
> > > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > > close system
> > > > > > > calls.
> > > > > > > > > >
> > > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > > >
> > > > > > > > > > We used time command to determine how much time the
> > > > > > > > > > system took
> > > > > to
> > > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > > (results attached
> > > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > > With-auditing-RAW)
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > System details:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 1 CPU machine
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > *OS Version*
> > > > > > > > > >
> > > > > > > > > > RHEL 6.5
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > *Kernel Version*
> > > > > > > > > >
> > > > > > > > > > uname –r
> > > > > > > > > >
> > > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > > for most
> > > > > of
> > > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Thanks & Regards,
> > > > > > > > > >
> > > > > > > > > > Logeswari.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Please Donate to
> > > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > > >
> > > > > > > > --
> > > > > > > > Linux-audit mailing list
> > > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > > >
> > > > > > >
> > > > > > > - RGB
> > > > > > >
> > > > > > > --
> > > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > > Canada
> > > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > > +1.613.693.0684x3545
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Please Donate to www.wikipedia.org
> > > > >
> > > > > - RGB
> > > > >
> > > > > --
> > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > +1.613.693.0684x3545
> > > > >
> > > >
> > > >
> > > > --
> > > > Please Donate to www.wikipedia.org
> > >
> > > - RGB
> > >
> > > --
> > > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > > Ottawa, Canada
> > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > +1.613.693.0684x3545
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <sys/stat.h>
> > #include <fcntl.h>
> > #include <unistd.h>
> > #include <errno.h>
> >
> > void create_load(int iters);
> > void cleanup();
> >
> > int high_rate = 0;
> > int num_iters = 100000;
> > int fd1;
> > char file1[50];
> > char file2[50];
> > char dir1[50];
> > char symlink1[50];
> >
> > /* Purpose: To create system load by invoking system calls used by templates.
> > *
> > * Note: The unlink(2) of a file can be an expensive operation (i.e., event
> > * rate goes way down).
> > */
> >
> > main(int argc, char **argv) {
> >
> > int num_children=1;
> > int iters;
> > int i;
> > char c;
> >
> > while ((c = getopt(argc, argv, "hi:")) != -1) {
> > switch (c) {
> > case 'h':
> > /*
> > * Desire "high" event rate
> > */
> > high_rate = 1;
> > argc--;
> > break;
> > case 'i':
> > /*
> > * Desire a specified number of iterations
> > */
> > num_iters = atoi(optarg);
> > argc--;
> > break;
> > default:
> > fprintf(stderr,"Unknown option: %c\n",optarg);
> > exit(1);
> > }
> > }
> >
> >
> > /*if(argv[optind] != NULL) {
> > num_children = atoi(argv[optind]);
> > } else {
> > num_children = 0;
> > }
> > Register cleanup routine */
> > fprintf(stderr,"Registering cleanup routine...\n");
> > if (atexit(cleanup) == -1) {
> > fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> > errno,strerror(errno));
> > exit(1);
> > }
> >
> >
> > /* fork child processes, if any requested */
> > for(i=1; i < num_children; i++) {
> > if(fork() == 0) {
> >
> > printf("child pid: %d\n",getpid());
> >
> > /* Setup file names based on child's pid */
> > sprintf(file1,"./file1_%d",getpid());
> > sprintf(file2,"./file2_%d",getpid());
> > sprintf(dir1,"./dir1_%d",getpid());
> > sprintf(symlink1,"./file1symlink_%d",getpid());
> >
> > /* each child creates load */
> > iters=0;
> > if (num_iters == -1) {
> > while(1) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > } else {
> > while(iters < num_iters) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > }
> > }
> > }
> >
> > /* Parent creates load also */
> > printf("parent pid: %d\n",getpid());
> >
> > /* Setup file names based on parent's pid */
> > sprintf(file1,"./file1_%d",getpid());
> > sprintf(file2,"./file2_%d",getpid());
> > sprintf(dir1,"./dir1_%d",getpid());
> > sprintf(symlink1,"./file1symlink_%d",getpid());
> >
> > iters=0;
> > if (num_iters == -1) {
> > while(1) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > } else {
> > while(iters < num_iters) {
> > create_load(iters);
> > iters++;
> > if( (iters % 1000) == 0) {
> > printf("pid %d iteration %d\n",getpid(),iters);
> > }
> > }
> > }
> >
> > } /* main */
> >
> >
> > void create_load(int iters) {
> >
> > int pid;
> > char *args[2];
> > struct stat stat_buf;
> >
> > fd1 = creat(file1,0x644);
> > if (fd1 == -1) {
> > fprintf(stderr,"pid %d: creat() returned error for file %s, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (close(fd1) == -1) {
> > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > fd1 = open(file1, O_RDWR, 0777);
> > if (fd1 == -1) {
> > fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > /* Chown this file to root instead of user ids so that we don't generate a
> > * non-owned alert when the file is truncated when invoking creat() again
> > * as root on an existing file owned by another user.
> > */
> > if (chown(file1,0,0) == -1) {
> > fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> > getpid(),0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (fchown(fd1,0,0) == -1) {
> > fprintf(stderr,"pid %d: fchown(%d,%d) returned error, errno=%d(%s)\n",
> > getpid(),0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> > fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> > fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> >
> > if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> > fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (ftruncate(fd1,7) == -1) {
> > fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (close(fd1) == -1) {
> > fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (truncate(file1,3) == -1) {
> > fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (rename(file1,file2) == -1) {
> > fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (rename(file2,file1) == -1) {
> > fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file2,file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (link(file1,file2) == -1) {
> > fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (symlink(file1,symlink1) == -1) {
> > fprintf(stderr,"pid %d: symlink(%s,%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (lchown(symlink1,0,0) == -1) {
> > fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,0,0,errno,strerror(errno));
> > exit(1);
> > }
> >
> > if (lstat(symlink1,&stat_buf) == -1) {
> > fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (stat(file1,&stat_buf) == -1) {
> > fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(file1) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),file1,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(file2) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),file2,errno,strerror(errno));
> > exit(1);
> > }
> > if (unlink(symlink1) == -1) {
> > fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> > getpid(),symlink1,errno,strerror(errno));
> > exit(1);
> > }
> > if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> > fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > if (rmdir(dir1) == -1) {
> > fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> >
> > /* Fork every 10000 iterations to not use up process resources too quickly */
> > if ( (iters % 10000) == 0) {
> > pid = fork();
> > if(pid == 0) {
> > fprintf(stderr,"child pid %d: fork!\n",getpid());
> > // child
> > args[0] = "/bin/ls";
> > args[1] = NULL;
> > close(1);
> > close(2);
> > execve(args[0], args, NULL);
> > fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> > getpid(),args[0],errno,strerror(errno));
> > _exit(1);
> > } else if (pid < 0) {
> > fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > } else {
> > fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
> > }
> >
> > pid = vfork();
> > if(pid == 0) {
> > args[0] = "/bin/pwd";
> > args[1] = NULL;
> > close(1);
> > close(2);
> > execv(args[0], args);
> > fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> > getpid(),args[0],errno,strerror(errno));
> > _exit(1);
> > } else if (pid < 0) {
> > fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> > getpid(),errno,strerror(errno));
> > exit(1);
> > }
> > }
> >
> > /* Make sure everything is cleaned up and deleted before returning */
> > cleanup();
> >
> > } /* create_load() */
> >
> > void cleanup() {
> > close(fd1);
> > unlink(file1);
> > unlink(file2);
> > unlink(symlink1);
> > unlink(dir1);
> > return;
> > }
>
> > --
> > Linux-audit mailing list
> > Linux-audit@redhat.com
> > https://www.redhat.com/mailman/listinfo/linux-audit
>
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
>
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-12 16:43 UTC (permalink / raw)
To: linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914165C5D1@G4W3222.americas.hpqcorp.net>
Further details of our investigation:
format_decode is invoked by vsnprintf which in turn is invoked by audit_log_vformat
memcpy is invoked by audit_log_vformat
number.isra.1 is invoked by vsnprintf which in turn is invoked by audit_log_vformat
vsnprintf is invoked by audit_log_vformat
Hence we believe by reducing the number of invocation of audit_log_vformat will increase the linux audit performance.
Call graph details of the perf data as follows:
11.33% loader1 [kernel.kallsyms] [k] format_decode
|
--- format_decode
|
|--89.02%-- vsnprintf
| audit_log_vformat
| audit_log_format
| |
| |--35.84%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--28.44%-- audit_log_start
| | |
| | |--65.65%-- audit_log_exit
| | | __audit_syscall_exit
| | --34.35%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--17.33%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| |--16.12%-- audit_log_exit
| | __audit_syscall_exit
| --2.28%-- audit_log_key
| audit_log_exit
| __audit_syscall_exit
--10.98%-- audit_log_vformat
audit_log_format
|
|--38.80%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
|--29.70%-- audit_log_start
| |
| |--86.87%-- audit_log_exit
| | __audit_syscall_exit
| --13.13%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
|--17.44%-- audit_log_exit
| __audit_syscall_exit
--14.06%-- audit_log_name
audit_log_exit
__audit_syscall_exit
10.40% loader1 [kernel.kallsyms] [k] memcpy
|
--- memcpy
|
|--94.81%-- audit_log_vformat
| audit_log_format
| |
| |--32.38%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--32.37%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| |--17.47%-- audit_log_start
| | |
| | |--66.60%-- audit_log_exit
| | | __audit_syscall_exit
| | --33.40%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--16.78%-- audit_log_exit
| | __audit_syscall_exit
| --1.00%-- audit_log_key
| audit_log_exit
| __audit_syscall_exit
--5.19%-- audit_log_n_untrustedstring
audit_log_untrustedstring
|
|--71.89%-- audit_log_d_path
| |
| |--50.90%-- audit_log_exit
| | __audit_syscall_exit
| --49.10%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
|--18.31%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
--9.81%-- audit_log_name
audit_log_exit
__audit_syscall_exit
7.46% loader1 [kernel.kallsyms] [k] number.isra.1
|
--- number.isra.1
|
|--96.98%-- vsnprintf
| audit_log_vformat
| audit_log_format
| |
| |--35.39%-- audit_log_start
| | |
| | |--66.68%-- audit_log_exit
| | | __audit_syscall_exit
| | --33.32%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--26.88%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--19.75%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| --17.98%-- audit_log_exit
| __audit_syscall_exit
--3.02%-- audit_log_vformat
audit_log_format
|
|--39.98%-- audit_log_start
| audit_log_name
| audit_log_exit
| __audit_syscall_exit
|--38.59%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
--21.42%-- audit_log_exit
__audit_syscall_exit
6.99% loader1 [kernel.kallsyms] [k] vsnprintf
|
--- vsnprintf
|
|--97.80%-- audit_log_vformat
| audit_log_format
| |
| |--28.93%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--26.83%-- audit_log_start
| | |
| | |--70.65%-- audit_log_exit
| | | __audit_syscall_exit
| | --29.35%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| |--25.86%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| |--14.33%-- audit_log_exit
| | __audit_syscall_exit
| |--2.55%-- audit_log_d_path
| | |
| | |--67.91%-- audit_log_task_info
| | | audit_log_exit
| | | __audit_syscall_exit
| | --32.09%-- audit_log_exit
| | __audit_syscall_exit
| |
| --1.50%-- audit_log_key
| audit_log_exit
| __audit_syscall_exit
--2.20%-- audit_log_format
|
|--66.67%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
--33.33%-- audit_log_key
audit_log_exit
__audit_syscall_exit
Regards,
Logeswari.
-----Original Message-----
From: linux-audit-bounces@redhat.com [mailto:linux-audit-bounces@redhat.com] On Behalf Of Viswanath, Logeswari P (MCOU OSTL)
Sent: Thursday, February 12, 2015 9:41 PM
To: linux-audit@redhat.com
Subject: RE: Linux audit performance impact
Hi all,
We did profiling of the kernel (using perf tool) during our performance test and below were the top 4 functions for the overhead.
11.33% loader1 [kernel.kallsyms] [k] format_decode
10.40% loader1 [kernel.kallsyms] [k] memcpy
7.46% loader1 [kernel.kallsyms] [k] number.isra.1
6.99% loader1 [kernel.kallsyms] [k] vsnprintf
I was unable to attach the entire profiling data of the kernel because it exceeds the limit of 80KB.
>From the perf data, we believed the overhead is because of invoking audit_log_format function multiple times.
We changed the code to reduce the number of times this function is called.
With this change the performance degradation is 20% now compared to the performance without auditing.
Without this change the performance degradation is 200% compared to the performance without auditing.
We can publish the code change done tomorrow.
Please let me know your feedback on this idea.
Regards,
Logeswari.
-----Original Message-----
From: Richard Guy Briggs [mailto:rgb@redhat.com]
Sent: Wednesday, February 11, 2015 10:21 PM
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: linux-audit@redhat.com
Subject: Re: Linux audit performance impact
On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Hi all,
>
> Please find the below the details of the performance test we ran.
> It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
>
> Kernel Version:
> root > uname -r
> 3.13.0-36-generic
>
> OS Version:
> Ubuntu 14.04.1
>
> No. of CPUs:
> root > nproc
> 24
>
> Audit Status:
> root > auditctl -s
> AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> lost=57190353 backlog=0
>
> Rules Configured:
> root > auditctl -l
> LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
>
> Attached is the program used to load the system.
>
> Results:
>
> Without enabling audit 12.29
> With auditing enabled and no rules configured 12.31
> With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
This would be more meaningful if you hacked the kernel to drain the queue figuratively to /dev/nul to eliminate the effect of auditd draining it, or syslog covering for a missing auditd. This stat doesn't tell us that much since the I/O act can vary significantly per installation. That one rule you chose is pretty unnaturally abusive and needs to be carefully thought out to avoid self-measurement.
> The degradation is around 200%
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 04, 2015 9:46 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > The intent is to calculate the performance impact by the auditing
> > components such as
> >
> > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
>
> Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
>
> > 2) impact because of running auditd - log format NOLOG
> > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Tuesday, February 03, 2015 10:33 PM
> > To: Satish Chandra Kilaru
> > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/03, Satish Chandra Kilaru wrote:
> > > Thanks for The info. But my question was rhetorical... I meant to
> > > say that it would not be much... She is trying to bombard the
> > > system with open calls ... So lots and lots of events will be
> > > generated and kernel has to write down the events some where or discard them...
> >
> > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> >
> > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > >
> > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > How many events can kernel accumulate without I/o ?
> > > >
> > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > set it to 320. It is now possible to set it to "0" which means
> > > > limited only by system resources. See "man auditctl", "-b"
> > > > option. An event can be made up of several buffers.
> > > >
> > > > Of course, how long a system lasts before the queue blows up
> > > > depends on your rule set...
> > > >
> > > > However, at the moment, it will still write out to klog if
> > > > auditd isn't running.
> > > >
> > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > >
> > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > collection),
> > > > > > but just do not want the records to delivered to user space
> > > > > > since I
> > > > want to
> > > > > > remove the I/O overhead while running the performance test.
> > > > > > Is there any option for this?
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > <javascript:;>
> > > > <javascript:;>]
> > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > linux-audit@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;>
> > > > > > Subject: Re: Linux audit performance impact
> > > > > >
> > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > Please read my question as “Is there any option to
> > > > > > > configure kaudit not to log audit records to syslog? when auditd not running.”
> > > > > >
> > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > audit=0 in
> > > > its
> > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > run since
> > > > boot.
> > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > to run
> > > > auditd.
> > > > > >
> > > > > > There is a feature request that is likely coming soon that
> > > > > > could be
> > > > > > useful:
> > > > > >
> > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > >
> > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > Subject: RE: Linux audit performance impact
> > > > > > >
> > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > records to
> > > > > > syslog when auditd is running?
> > > > > > > This way we can assess the impact of enabling audit
> > > > > > > without involving
> > > > > > disk I/o overhead.
> > > > > > >
> > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > <javascript:;> <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > To: Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > linux-audit@redhat.com <javascript:;>
> > > > > > <javascript:;>>; Viswanath,
> > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > further events
> > > > > > leading spiralling of events...
> > > > > > > I brought down my server few times with stupid rules...
> > > > > > >
> > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > <sgrubb@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > <javascript:;>>> wrote:
> > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > Kilaru
> > > > wrote:
> > > > > > > > Write your own program to receive audit events directly
> > > > > > > > without using auditd...
> > > > > > > > That should be faster ....
> > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > >
> > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > if you
> > > > want
> > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > made it to disk and the priority of that audit daemon.
> > > > > > > Then you also have all
> > > > the
> > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > for any disk performance critical system.
> > > > > > >
> > > > > > > -Steve
> > > > > > >
> > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > (MCOU
> > > > > > > > OSTL)
> > > > <
> > > > > > > >
> > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > logeswari.pv@hp.com <javascript:;>
> > > > > > <javascript:;>>> wrote:
> > > > > > > > > Hi Steve,
> > > > > > > > >
> > > > > > > > > I am Logeswari working for HP.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > Suse linux
> > > > to
> > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > host based
> > > > IDS.
> > > > > > > > >
> > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > system calls auditing is enabled for open/close system
> > > > > > > > > call which is a HUGE impact on the system performance.
> > > > > > > > > It would
> > > > be
> > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > reason
> > > > > > > > > behind it and can we fix it?
> > > > > > > > >
> > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > impact? If
> > > > > > yes,
> > > > > > > > > can you please share the numbers and also the
> > > > > > > > > steps/programs used the run the same.
> > > > > > > > >
> > > > > > > > > 3) Help us validating the performance test we have done in
> > > > our
> > > > > > test
> > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > close system
> > > > > > calls.
> > > > > > > > >
> > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > >
> > > > > > > > > We used time command to determine how much time the
> > > > > > > > > system took
> > > > to
> > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > (results attached
> > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > With-auditing-RAW)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > System details:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1 CPU machine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *OS Version*
> > > > > > > > >
> > > > > > > > > RHEL 6.5
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *Kernel Version*
> > > > > > > > >
> > > > > > > > > uname –r
> > > > > > > > >
> > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > for most
> > > > of
> > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks & Regards,
> > > > > > > > >
> > > > > > > > > Logeswari.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Please Donate to
> > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > >
> > > > > > > --
> > > > > > > Linux-audit mailing list
> > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > >
> > > > > >
> > > > > > - RGB
> > > > > >
> > > > > > --
> > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > Canada
> > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > +1.613.693.0684x3545
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Please Donate to www.wikipedia.org
> > > >
> > > > - RGB
> > > >
> > > > --
> > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > +1.613.693.0684x3545
> > > >
> > >
> > >
> > > --
> > > Please Donate to www.wikipedia.org
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> +1.613.693.0684x3545
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <errno.h>
>
> void create_load(int iters);
> void cleanup();
>
> int high_rate = 0;
> int num_iters = 100000;
> int fd1;
> char file1[50];
> char file2[50];
> char dir1[50];
> char symlink1[50];
>
> /* Purpose: To create system load by invoking system calls used by templates.
> *
> * Note: The unlink(2) of a file can be an expensive operation (i.e., event
> * rate goes way down).
> */
>
> main(int argc, char **argv) {
>
> int num_children=1;
> int iters;
> int i;
> char c;
>
> while ((c = getopt(argc, argv, "hi:")) != -1) {
> switch (c) {
> case 'h':
> /*
> * Desire "high" event rate
> */
> high_rate = 1;
> argc--;
> break;
> case 'i':
> /*
> * Desire a specified number of iterations
> */
> num_iters = atoi(optarg);
> argc--;
> break;
> default:
> fprintf(stderr,"Unknown option: %c\n",optarg);
> exit(1);
> }
> }
>
>
> /*if(argv[optind] != NULL) {
> num_children = atoi(argv[optind]);
> } else {
> num_children = 0;
> }
> Register cleanup routine */
> fprintf(stderr,"Registering cleanup routine...\n");
> if (atexit(cleanup) == -1) {
> fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> errno,strerror(errno));
> exit(1);
> }
>
>
> /* fork child processes, if any requested */
> for(i=1; i < num_children; i++) {
> if(fork() == 0) {
>
> printf("child pid: %d\n",getpid());
>
> /* Setup file names based on child's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> /* each child creates load */
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
> }
> }
>
> /* Parent creates load also */
> printf("parent pid: %d\n",getpid());
>
> /* Setup file names based on parent's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
>
> } /* main */
>
>
> void create_load(int iters) {
>
> int pid;
> char *args[2];
> struct stat stat_buf;
>
> fd1 = creat(file1,0x644);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: creat() returned error for file %s, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> fd1 = open(file1, O_RDWR, 0777);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Chown this file to root instead of user ids so that we don't generate a
> * non-owned alert when the file is truncated when invoking creat() again
> * as root on an existing file owned by another user.
> */
> if (chown(file1,0,0) == -1) {
> fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (fchown(fd1,0,0) == -1) {
> fprintf(stderr,"pid %d: fchown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
>
> if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (ftruncate(fd1,7) == -1) {
> fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> if (truncate(file1,3) == -1) {
> fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rename(file1,file2) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (rename(file2,file1) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file2,file1,errno,strerror(errno));
> exit(1);
> }
> if (link(file1,file2) == -1) {
> fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (symlink(file1,symlink1) == -1) {
> fprintf(stderr,"pid %d: symlink(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,symlink1,errno,strerror(errno));
> exit(1);
> }
> if (lchown(symlink1,0,0) == -1) {
> fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error, errno=%d(%s)\n",
> getpid(),symlink1,0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (lstat(symlink1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (stat(file1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file2) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file2,errno,strerror(errno));
> exit(1);
> }
> if (unlink(symlink1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rmdir(dir1) == -1) {
> fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Fork every 10000 iterations to not use up process resources too quickly */
> if ( (iters % 10000) == 0) {
> pid = fork();
> if(pid == 0) {
> fprintf(stderr,"child pid %d: fork!\n",getpid());
> // child
> args[0] = "/bin/ls";
> args[1] = NULL;
> close(1);
> close(2);
> execve(args[0], args, NULL);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> } else {
> fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
> }
>
> pid = vfork();
> if(pid == 0) {
> args[0] = "/bin/pwd";
> args[1] = NULL;
> close(1);
> close(2);
> execv(args[0], args);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> }
>
> /* Make sure everything is cleaned up and deleted before returning */
> cleanup();
>
> } /* create_load() */
>
> void cleanup() {
> close(fd1);
> unlink(file1);
> unlink(file2);
> unlink(symlink1);
> unlink(dir1);
> return;
> }
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* Re: Linux audit performance impact
From: Paul Moore @ 2015-02-12 16:31 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL); +Cc: linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914165C5D1@G4W3222.americas.hpqcorp.net>
On Thu, Feb 12, 2015 at 11:10 AM, Viswanath, Logeswari P (MCOU OSTL)
<logeswari.pv@hp.com> wrote:
> Hi all,
>
> We did profiling of the kernel (using perf tool) during our performance test and below were the top 4 functions for the overhead.
>
> 11.33% loader1 [kernel.kallsyms] [k] format_decode
> 10.40% loader1 [kernel.kallsyms] [k] memcpy
> 7.46% loader1 [kernel.kallsyms] [k] number.isra.1
> 6.99% loader1 [kernel.kallsyms] [k] vsnprintf
>
> I was unable to attach the entire profiling data of the kernel because it exceeds the limit of 80KB.
>
> >From the perf data, we believed the overhead is because of invoking audit_log_format function multiple times.
> We changed the code to reduce the number of times this function is called.
> With this change the performance degradation is 20% now compared to the performance without auditing.
> Without this change the performance degradation is 200% compared to the performance without auditing.
>
> We can publish the code change done tomorrow.
>
> Please let me know your feedback on this idea.
This doesn't surprise me, this due to the string based record format -
it's expense to generate those strings. I'd be interested in seeing
your patches.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-12 16:10 UTC (permalink / raw)
To: linux-audit@redhat.com
Hi all,
We did profiling of the kernel (using perf tool) during our performance test and below were the top 4 functions for the overhead.
11.33% loader1 [kernel.kallsyms] [k] format_decode
10.40% loader1 [kernel.kallsyms] [k] memcpy
7.46% loader1 [kernel.kallsyms] [k] number.isra.1
6.99% loader1 [kernel.kallsyms] [k] vsnprintf
I was unable to attach the entire profiling data of the kernel because it exceeds the limit of 80KB.
>From the perf data, we believed the overhead is because of invoking audit_log_format function multiple times.
We changed the code to reduce the number of times this function is called.
With this change the performance degradation is 20% now compared to the performance without auditing.
Without this change the performance degradation is 200% compared to the performance without auditing.
We can publish the code change done tomorrow.
Please let me know your feedback on this idea.
Regards,
Logeswari.
-----Original Message-----
From: Richard Guy Briggs [mailto:rgb@redhat.com]
Sent: Wednesday, February 11, 2015 10:21 PM
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: linux-audit@redhat.com
Subject: Re: Linux audit performance impact
On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Hi all,
>
> Please find the below the details of the performance test we ran.
> It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
>
> Kernel Version:
> root > uname -r
> 3.13.0-36-generic
>
> OS Version:
> Ubuntu 14.04.1
>
> No. of CPUs:
> root > nproc
> 24
>
> Audit Status:
> root > auditctl -s
> AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> lost=57190353 backlog=0
>
> Rules Configured:
> root > auditctl -l
> LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
>
> Attached is the program used to load the system.
>
> Results:
>
> Without enabling audit 12.29
> With auditing enabled and no rules configured 12.31
> With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
This would be more meaningful if you hacked the kernel to drain the queue figuratively to /dev/nul to eliminate the effect of auditd draining it, or syslog covering for a missing auditd. This stat doesn't tell us that much since the I/O act can vary significantly per installation. That one rule you chose is pretty unnaturally abusive and needs to be carefully thought out to avoid self-measurement.
> The degradation is around 200%
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 04, 2015 9:46 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > The intent is to calculate the performance impact by the auditing
> > components such as
> >
> > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
>
> Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
>
> > 2) impact because of running auditd - log format NOLOG
> > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Tuesday, February 03, 2015 10:33 PM
> > To: Satish Chandra Kilaru
> > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/03, Satish Chandra Kilaru wrote:
> > > Thanks for The info. But my question was rhetorical... I meant to
> > > say that it would not be much... She is trying to bombard the
> > > system with open calls ... So lots and lots of events will be
> > > generated and kernel has to write down the events some where or discard them...
> >
> > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> >
> > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > >
> > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > How many events can kernel accumulate without I/o ?
> > > >
> > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > set it to 320. It is now possible to set it to "0" which means
> > > > limited only by system resources. See "man auditctl", "-b"
> > > > option. An event can be made up of several buffers.
> > > >
> > > > Of course, how long a system lasts before the queue blows up
> > > > depends on your rule set...
> > > >
> > > > However, at the moment, it will still write out to klog if
> > > > auditd isn't running.
> > > >
> > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > >
> > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > collection),
> > > > > > but just do not want the records to delivered to user space
> > > > > > since I
> > > > want to
> > > > > > remove the I/O overhead while running the performance test.
> > > > > > Is there any option for this?
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > <javascript:;>
> > > > <javascript:;>]
> > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > linux-audit@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;>
> > > > > > Subject: Re: Linux audit performance impact
> > > > > >
> > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > Please read my question as “Is there any option to
> > > > > > > configure kaudit not to log audit records to syslog? when auditd not running.”
> > > > > >
> > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > audit=0 in
> > > > its
> > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > run since
> > > > boot.
> > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > to run
> > > > auditd.
> > > > > >
> > > > > > There is a feature request that is likely coming soon that
> > > > > > could be
> > > > > > useful:
> > > > > >
> > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > >
> > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > Subject: RE: Linux audit performance impact
> > > > > > >
> > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > records to
> > > > > > syslog when auditd is running?
> > > > > > > This way we can assess the impact of enabling audit
> > > > > > > without involving
> > > > > > disk I/o overhead.
> > > > > > >
> > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > <javascript:;> <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > To: Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > linux-audit@redhat.com <javascript:;>
> > > > > > <javascript:;>>; Viswanath,
> > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > further events
> > > > > > leading spiralling of events...
> > > > > > > I brought down my server few times with stupid rules...
> > > > > > >
> > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > <sgrubb@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > <javascript:;>>> wrote:
> > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > Kilaru
> > > > wrote:
> > > > > > > > Write your own program to receive audit events directly
> > > > > > > > without using auditd...
> > > > > > > > That should be faster ....
> > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > >
> > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > if you
> > > > want
> > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > made it to disk and the priority of that audit daemon.
> > > > > > > Then you also have all
> > > > the
> > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > for any disk performance critical system.
> > > > > > >
> > > > > > > -Steve
> > > > > > >
> > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > (MCOU
> > > > > > > > OSTL)
> > > > <
> > > > > > > >
> > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > logeswari.pv@hp.com <javascript:;>
> > > > > > <javascript:;>>> wrote:
> > > > > > > > > Hi Steve,
> > > > > > > > >
> > > > > > > > > I am Logeswari working for HP.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > Suse linux
> > > > to
> > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > host based
> > > > IDS.
> > > > > > > > >
> > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > system calls auditing is enabled for open/close system
> > > > > > > > > call which is a HUGE impact on the system performance.
> > > > > > > > > It would
> > > > be
> > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > reason
> > > > > > > > > behind it and can we fix it?
> > > > > > > > >
> > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > impact? If
> > > > > > yes,
> > > > > > > > > can you please share the numbers and also the
> > > > > > > > > steps/programs used the run the same.
> > > > > > > > >
> > > > > > > > > 3) Help us validating the performance test we have done in
> > > > our
> > > > > > test
> > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > close system
> > > > > > calls.
> > > > > > > > >
> > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > >
> > > > > > > > > We used time command to determine how much time the
> > > > > > > > > system took
> > > > to
> > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > (results attached
> > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > With-auditing-RAW)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > System details:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1 CPU machine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *OS Version*
> > > > > > > > >
> > > > > > > > > RHEL 6.5
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *Kernel Version*
> > > > > > > > >
> > > > > > > > > uname –r
> > > > > > > > >
> > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > for most
> > > > of
> > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks & Regards,
> > > > > > > > >
> > > > > > > > > Logeswari.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Please Donate to
> > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > >
> > > > > > > --
> > > > > > > Linux-audit mailing list
> > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > >
> > > > > >
> > > > > > - RGB
> > > > > >
> > > > > > --
> > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > Canada
> > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > +1.613.693.0684x3545
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Please Donate to www.wikipedia.org
> > > >
> > > > - RGB
> > > >
> > > > --
> > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > +1.613.693.0684x3545
> > > >
> > >
> > >
> > > --
> > > Please Donate to www.wikipedia.org
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> +1.613.693.0684x3545
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <errno.h>
>
> void create_load(int iters);
> void cleanup();
>
> int high_rate = 0;
> int num_iters = 100000;
> int fd1;
> char file1[50];
> char file2[50];
> char dir1[50];
> char symlink1[50];
>
> /* Purpose: To create system load by invoking system calls used by templates.
> *
> * Note: The unlink(2) of a file can be an expensive operation (i.e., event
> * rate goes way down).
> */
>
> main(int argc, char **argv) {
>
> int num_children=1;
> int iters;
> int i;
> char c;
>
> while ((c = getopt(argc, argv, "hi:")) != -1) {
> switch (c) {
> case 'h':
> /*
> * Desire "high" event rate
> */
> high_rate = 1;
> argc--;
> break;
> case 'i':
> /*
> * Desire a specified number of iterations
> */
> num_iters = atoi(optarg);
> argc--;
> break;
> default:
> fprintf(stderr,"Unknown option: %c\n",optarg);
> exit(1);
> }
> }
>
>
> /*if(argv[optind] != NULL) {
> num_children = atoi(argv[optind]);
> } else {
> num_children = 0;
> }
> Register cleanup routine */
> fprintf(stderr,"Registering cleanup routine...\n");
> if (atexit(cleanup) == -1) {
> fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> errno,strerror(errno));
> exit(1);
> }
>
>
> /* fork child processes, if any requested */
> for(i=1; i < num_children; i++) {
> if(fork() == 0) {
>
> printf("child pid: %d\n",getpid());
>
> /* Setup file names based on child's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> /* each child creates load */
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
> }
> }
>
> /* Parent creates load also */
> printf("parent pid: %d\n",getpid());
>
> /* Setup file names based on parent's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
>
> } /* main */
>
>
> void create_load(int iters) {
>
> int pid;
> char *args[2];
> struct stat stat_buf;
>
> fd1 = creat(file1,0x644);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: creat() returned error for file %s, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> fd1 = open(file1, O_RDWR, 0777);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Chown this file to root instead of user ids so that we don't generate a
> * non-owned alert when the file is truncated when invoking creat() again
> * as root on an existing file owned by another user.
> */
> if (chown(file1,0,0) == -1) {
> fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (fchown(fd1,0,0) == -1) {
> fprintf(stderr,"pid %d: fchown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
>
> if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (ftruncate(fd1,7) == -1) {
> fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> if (truncate(file1,3) == -1) {
> fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rename(file1,file2) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (rename(file2,file1) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file2,file1,errno,strerror(errno));
> exit(1);
> }
> if (link(file1,file2) == -1) {
> fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (symlink(file1,symlink1) == -1) {
> fprintf(stderr,"pid %d: symlink(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,symlink1,errno,strerror(errno));
> exit(1);
> }
> if (lchown(symlink1,0,0) == -1) {
> fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error, errno=%d(%s)\n",
> getpid(),symlink1,0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (lstat(symlink1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (stat(file1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file2) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file2,errno,strerror(errno));
> exit(1);
> }
> if (unlink(symlink1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rmdir(dir1) == -1) {
> fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Fork every 10000 iterations to not use up process resources too quickly */
> if ( (iters % 10000) == 0) {
> pid = fork();
> if(pid == 0) {
> fprintf(stderr,"child pid %d: fork!\n",getpid());
> // child
> args[0] = "/bin/ls";
> args[1] = NULL;
> close(1);
> close(2);
> execve(args[0], args, NULL);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> } else {
> fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
> }
>
> pid = vfork();
> if(pid == 0) {
> args[0] = "/bin/pwd";
> args[1] = NULL;
> close(1);
> close(2);
> execv(args[0], args);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> }
>
> /* Make sure everything is cleaned up and deleted before returning */
> cleanup();
>
> } /* create_load() */
>
> void cleanup() {
> close(fd1);
> unlink(file1);
> unlink(file2);
> unlink(symlink1);
> unlink(dir1);
> return;
> }
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* Re: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-12 16:09 UTC (permalink / raw)
To: Richard Guy Briggs; +Cc: linux-audit@redhat.com
Hi all,
We did profiling of the kernel (using perf tool) during our performance test and below were the top 4 functions for the overhead.
11.33% loader1 [kernel.kallsyms] [k] format_decode
10.40% loader1 [kernel.kallsyms] [k] memcpy
7.46% loader1 [kernel.kallsyms] [k] number.isra.1
6.99% loader1 [kernel.kallsyms] [k] vsnprintf
I was unable to attach the entire profiling data of the kernel because it exceeds the limit of 80KB.
>From the perf data, we believed the overhead is because of invoking audit_log_format function multiple times.
We changed the code to reduce the number of times this function is called.
With this change the performance degradation is 20% now compared to the performance without auditing.
Without this change the performance degradation is 200% compared to the performance without auditing.
We can publish the code change done tomorrow.
Please let me know your feedback on this idea.
Regards,
Logeswari.
-----Original Message-----
From: Richard Guy Briggs [mailto:rgb@redhat.com]
Sent: Wednesday, February 11, 2015 10:21 PM
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: linux-audit@redhat.com
Subject: Re: Linux audit performance impact
On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Hi all,
>
> Please find the below the details of the performance test we ran.
> It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
>
> Kernel Version:
> root > uname -r
> 3.13.0-36-generic
>
> OS Version:
> Ubuntu 14.04.1
>
> No. of CPUs:
> root > nproc
> 24
>
> Audit Status:
> root > auditctl -s
> AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> lost=57190353 backlog=0
>
> Rules Configured:
> root > auditctl -l
> LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
>
> Attached is the program used to load the system.
>
> Results:
>
> Without enabling audit 12.29
> With auditing enabled and no rules configured 12.31
> With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
This would be more meaningful if you hacked the kernel to drain the queue figuratively to /dev/nul to eliminate the effect of auditd draining it, or syslog covering for a missing auditd. This stat doesn't tell us that much since the I/O act can vary significantly per installation. That one rule you chose is pretty unnaturally abusive and needs to be carefully thought out to avoid self-measurement.
> The degradation is around 200%
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 04, 2015 9:46 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > The intent is to calculate the performance impact by the auditing
> > components such as
> >
> > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
>
> Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
>
> > 2) impact because of running auditd - log format NOLOG
> > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Tuesday, February 03, 2015 10:33 PM
> > To: Satish Chandra Kilaru
> > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/03, Satish Chandra Kilaru wrote:
> > > Thanks for The info. But my question was rhetorical... I meant to
> > > say that it would not be much... She is trying to bombard the
> > > system with open calls ... So lots and lots of events will be
> > > generated and kernel has to write down the events some where or discard them...
> >
> > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> >
> > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > >
> > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > How many events can kernel accumulate without I/o ?
> > > >
> > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > set it to 320. It is now possible to set it to "0" which means
> > > > limited only by system resources. See "man auditctl", "-b"
> > > > option. An event can be made up of several buffers.
> > > >
> > > > Of course, how long a system lasts before the queue blows up
> > > > depends on your rule set...
> > > >
> > > > However, at the moment, it will still write out to klog if
> > > > auditd isn't running.
> > > >
> > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > >
> > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > collection),
> > > > > > but just do not want the records to delivered to user space
> > > > > > since I
> > > > want to
> > > > > > remove the I/O overhead while running the performance test.
> > > > > > Is there any option for this?
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > <javascript:;>
> > > > <javascript:;>]
> > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > linux-audit@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;>
> > > > > > Subject: Re: Linux audit performance impact
> > > > > >
> > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > Please read my question as “Is there any option to
> > > > > > > configure kaudit not to log audit records to syslog? when auditd not running.”
> > > > > >
> > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > audit=0 in
> > > > its
> > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > run since
> > > > boot.
> > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > to run
> > > > auditd.
> > > > > >
> > > > > > There is a feature request that is likely coming soon that
> > > > > > could be
> > > > > > useful:
> > > > > >
> > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > >
> > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > Subject: RE: Linux audit performance impact
> > > > > > >
> > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > records to
> > > > > > syslog when auditd is running?
> > > > > > > This way we can assess the impact of enabling audit
> > > > > > > without involving
> > > > > > disk I/o overhead.
> > > > > > >
> > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > <javascript:;> <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > To: Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > linux-audit@redhat.com <javascript:;>
> > > > > > <javascript:;>>; Viswanath,
> > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > further events
> > > > > > leading spiralling of events...
> > > > > > > I brought down my server few times with stupid rules...
> > > > > > >
> > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > <sgrubb@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > <javascript:;>>> wrote:
> > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > Kilaru
> > > > wrote:
> > > > > > > > Write your own program to receive audit events directly
> > > > > > > > without using auditd...
> > > > > > > > That should be faster ....
> > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > >
> > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > if you
> > > > want
> > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > made it to disk and the priority of that audit daemon.
> > > > > > > Then you also have all
> > > > the
> > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > for any disk performance critical system.
> > > > > > >
> > > > > > > -Steve
> > > > > > >
> > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > (MCOU
> > > > > > > > OSTL)
> > > > <
> > > > > > > >
> > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > logeswari.pv@hp.com <javascript:;>
> > > > > > <javascript:;>>> wrote:
> > > > > > > > > Hi Steve,
> > > > > > > > >
> > > > > > > > > I am Logeswari working for HP.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > Suse linux
> > > > to
> > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > host based
> > > > IDS.
> > > > > > > > >
> > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > system calls auditing is enabled for open/close system
> > > > > > > > > call which is a HUGE impact on the system performance.
> > > > > > > > > It would
> > > > be
> > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > reason
> > > > > > > > > behind it and can we fix it?
> > > > > > > > >
> > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > impact? If
> > > > > > yes,
> > > > > > > > > can you please share the numbers and also the
> > > > > > > > > steps/programs used the run the same.
> > > > > > > > >
> > > > > > > > > 3) Help us validating the performance test we have done in
> > > > our
> > > > > > test
> > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > close system
> > > > > > calls.
> > > > > > > > >
> > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > >
> > > > > > > > > We used time command to determine how much time the
> > > > > > > > > system took
> > > > to
> > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > (results attached
> > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > With-auditing-RAW)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > System details:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1 CPU machine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *OS Version*
> > > > > > > > >
> > > > > > > > > RHEL 6.5
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *Kernel Version*
> > > > > > > > >
> > > > > > > > > uname –r
> > > > > > > > >
> > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > for most
> > > > of
> > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks & Regards,
> > > > > > > > >
> > > > > > > > > Logeswari.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Please Donate to
> > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > >
> > > > > > > --
> > > > > > > Linux-audit mailing list
> > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > >
> > > > > >
> > > > > > - RGB
> > > > > >
> > > > > > --
> > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > Canada
> > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > +1.613.693.0684x3545
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Please Donate to www.wikipedia.org
> > > >
> > > > - RGB
> > > >
> > > > --
> > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > +1.613.693.0684x3545
> > > >
> > >
> > >
> > > --
> > > Please Donate to www.wikipedia.org
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> +1.613.693.0684x3545
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <errno.h>
>
> void create_load(int iters);
> void cleanup();
>
> int high_rate = 0;
> int num_iters = 100000;
> int fd1;
> char file1[50];
> char file2[50];
> char dir1[50];
> char symlink1[50];
>
> /* Purpose: To create system load by invoking system calls used by templates.
> *
> * Note: The unlink(2) of a file can be an expensive operation (i.e., event
> * rate goes way down).
> */
>
> main(int argc, char **argv) {
>
> int num_children=1;
> int iters;
> int i;
> char c;
>
> while ((c = getopt(argc, argv, "hi:")) != -1) {
> switch (c) {
> case 'h':
> /*
> * Desire "high" event rate
> */
> high_rate = 1;
> argc--;
> break;
> case 'i':
> /*
> * Desire a specified number of iterations
> */
> num_iters = atoi(optarg);
> argc--;
> break;
> default:
> fprintf(stderr,"Unknown option: %c\n",optarg);
> exit(1);
> }
> }
>
>
> /*if(argv[optind] != NULL) {
> num_children = atoi(argv[optind]);
> } else {
> num_children = 0;
> }
> Register cleanup routine */
> fprintf(stderr,"Registering cleanup routine...\n");
> if (atexit(cleanup) == -1) {
> fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> errno,strerror(errno));
> exit(1);
> }
>
>
> /* fork child processes, if any requested */
> for(i=1; i < num_children; i++) {
> if(fork() == 0) {
>
> printf("child pid: %d\n",getpid());
>
> /* Setup file names based on child's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> /* each child creates load */
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
> }
> }
>
> /* Parent creates load also */
> printf("parent pid: %d\n",getpid());
>
> /* Setup file names based on parent's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
>
> } /* main */
>
>
> void create_load(int iters) {
>
> int pid;
> char *args[2];
> struct stat stat_buf;
>
> fd1 = creat(file1,0x644);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: creat() returned error for file %s, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> fd1 = open(file1, O_RDWR, 0777);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Chown this file to root instead of user ids so that we don't generate a
> * non-owned alert when the file is truncated when invoking creat() again
> * as root on an existing file owned by another user.
> */
> if (chown(file1,0,0) == -1) {
> fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (fchown(fd1,0,0) == -1) {
> fprintf(stderr,"pid %d: fchown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
>
> if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (ftruncate(fd1,7) == -1) {
> fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> if (truncate(file1,3) == -1) {
> fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rename(file1,file2) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (rename(file2,file1) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file2,file1,errno,strerror(errno));
> exit(1);
> }
> if (link(file1,file2) == -1) {
> fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (symlink(file1,symlink1) == -1) {
> fprintf(stderr,"pid %d: symlink(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,symlink1,errno,strerror(errno));
> exit(1);
> }
> if (lchown(symlink1,0,0) == -1) {
> fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error, errno=%d(%s)\n",
> getpid(),symlink1,0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (lstat(symlink1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (stat(file1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file2) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file2,errno,strerror(errno));
> exit(1);
> }
> if (unlink(symlink1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rmdir(dir1) == -1) {
> fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Fork every 10000 iterations to not use up process resources too quickly */
> if ( (iters % 10000) == 0) {
> pid = fork();
> if(pid == 0) {
> fprintf(stderr,"child pid %d: fork!\n",getpid());
> // child
> args[0] = "/bin/ls";
> args[1] = NULL;
> close(1);
> close(2);
> execve(args[0], args, NULL);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> } else {
> fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
> }
>
> pid = vfork();
> if(pid == 0) {
> args[0] = "/bin/pwd";
> args[1] = NULL;
> close(1);
> close(2);
> execv(args[0], args);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> }
>
> /* Make sure everything is cleaned up and deleted before returning */
> cleanup();
>
> } /* create_load() */
>
> void cleanup() {
> close(fd1);
> unlink(file1);
> unlink(file2);
> unlink(symlink1);
> unlink(dir1);
> return;
> }
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-12 14:58 UTC (permalink / raw)
To: Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <20150211165108.GW29998@madcap2.tricolour.ca>
[-- Attachment #1: Type: text/plain, Size: 25045 bytes --]
Hi all,
We did profiling of the kernel during our performance test and below were the top 4 functions for the overhead.
11.33% loader1 [kernel.kallsyms] [k] format_decode
10.40% loader1 [kernel.kallsyms] [k] memcpy
7.46% loader1 [kernel.kallsyms] [k] number.isra.1
6.99% loader1 [kernel.kallsyms] [k] vsnprintf
Please find attached the complete profiling data of the kernel using perf tool.
From the perf data, we believed the overhead is because of invoking audit_log_format function multiple times.
We changed the code to reduce the number of times this function is called.
With this change the performance degradation is 20% now compared to the performance without auditing.
Without this change the performance degradation is 200% compared to the performance without auditing.
We can publish the code change done tomorrow.
Please let me know your feedback on this idea.
Regards,
Logeswari.
-----Original Message-----
From: Richard Guy Briggs [mailto:rgb@redhat.com]
Sent: Wednesday, February 11, 2015 10:21 PM
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: linux-audit@redhat.com
Subject: Re: Linux audit performance impact
On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Hi all,
>
> Please find the below the details of the performance test we ran.
> It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
>
> Kernel Version:
> root > uname -r
> 3.13.0-36-generic
>
> OS Version:
> Ubuntu 14.04.1
>
> No. of CPUs:
> root > nproc
> 24
>
> Audit Status:
> root > auditctl -s
> AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320
> lost=57190353 backlog=0
>
> Rules Configured:
> root > auditctl -l
> LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
>
> Attached is the program used to load the system.
>
> Results:
>
> Without enabling audit 12.29
> With auditing enabled and no rules configured 12.31
> With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
This would be more meaningful if you hacked the kernel to drain the queue figuratively to /dev/nul to eliminate the effect of auditd draining it, or syslog covering for a missing auditd. This stat doesn't tell us that much since the I/O act can vary significantly per installation. That one rule you chose is pretty unnaturally abusive and needs to be carefully thought out to avoid self-measurement.
> The degradation is around 200%
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 04, 2015 9:46 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > The intent is to calculate the performance impact by the auditing
> > components such as
> >
> > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
>
> Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
>
> > 2) impact because of running auditd - log format NOLOG
> > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Tuesday, February 03, 2015 10:33 PM
> > To: Satish Chandra Kilaru
> > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/03, Satish Chandra Kilaru wrote:
> > > Thanks for The info. But my question was rhetorical... I meant to
> > > say that it would not be much... She is trying to bombard the
> > > system with open calls ... So lots and lots of events will be
> > > generated and kernel has to write down the events some where or discard them...
> >
> > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> >
> > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > >
> > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > How many events can kernel accumulate without I/o ?
> > > >
> > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > set it to 320. It is now possible to set it to "0" which means
> > > > limited only by system resources. See "man auditctl", "-b"
> > > > option. An event can be made up of several buffers.
> > > >
> > > > Of course, how long a system lasts before the queue blows up
> > > > depends on your rule set...
> > > >
> > > > However, at the moment, it will still write out to klog if
> > > > auditd isn't running.
> > > >
> > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU
> > > > > OSTL) < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > >
> > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > collection),
> > > > > > but just do not want the records to delivered to user space
> > > > > > since I
> > > > want to
> > > > > > remove the I/O overhead while running the performance test.
> > > > > > Is there any option for this?
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com
> > > > > > <javascript:;>
> > > > <javascript:;>]
> > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > Cc: Satish Chandra Kilaru; Steve Grubb;
> > > > > > linux-audit@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;>
> > > > > > Subject: Re: Linux audit performance impact
> > > > > >
> > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > Please read my question as “Is there any option to
> > > > > > > configure kaudit not to log audit records to syslog? when auditd not running.”
> > > > > >
> > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > audit=0 in
> > > > its
> > > > > > place. This will stop all but AVCs and if auditd has ever
> > > > > > run since
> > > > boot.
> > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > to run
> > > > auditd.
> > > > > >
> > > > > > There is a feature request that is likely coming soon that
> > > > > > could be
> > > > > > useful:
> > > > > >
> > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > >
> > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > Subject: RE: Linux audit performance impact
> > > > > > >
> > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > records to
> > > > > > syslog when auditd is running?
> > > > > > > This way we can assess the impact of enabling audit
> > > > > > > without involving
> > > > > > disk I/o overhead.
> > > > > > >
> > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > <javascript:;> <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > To: Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > linux-audit@redhat.com <javascript:;>
> > > > > > <javascript:;>>; Viswanath,
> > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > I agree with you... but writing to disk can trigger
> > > > > > > further events
> > > > > > leading spiralling of events...
> > > > > > > I brought down my server few times with stupid rules...
> > > > > > >
> > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > <sgrubb@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > <javascript:;>>> wrote:
> > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > Kilaru
> > > > wrote:
> > > > > > > > Write your own program to receive audit events directly
> > > > > > > > without using auditd...
> > > > > > > > That should be faster ....
> > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > >
> > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > if you
> > > > want
> > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > made it to disk and the priority of that audit daemon.
> > > > > > > Then you also have all
> > > > the
> > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > for any disk performance critical system.
> > > > > > >
> > > > > > > -Steve
> > > > > > >
> > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > (MCOU
> > > > > > > > OSTL)
> > > > <
> > > > > > > >
> > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > logeswari.pv@hp.com <javascript:;>
> > > > > > <javascript:;>>> wrote:
> > > > > > > > > Hi Steve,
> > > > > > > > >
> > > > > > > > > I am Logeswari working for HP.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > Suse linux
> > > > to
> > > > > > > > > help us evaluate linux audit as data source for our
> > > > > > > > > host based
> > > > IDS.
> > > > > > > > >
> > > > > > > > > When we ran our own performance test with a test
> > > > > > > > > audispd plugin, we found if a system can perform
> > > > > > > > > 200000 open/close system calls per second without
> > > > > > > > > auditing, system can perform only 3000 open/close
> > > > > > > > > system calls auditing is enabled for open/close system
> > > > > > > > > call which is a HUGE impact on the system performance.
> > > > > > > > > It would
> > > > be
> > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > reason
> > > > > > > > > behind it and can we fix it?
> > > > > > > > >
> > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > impact? If
> > > > > > yes,
> > > > > > > > > can you please share the numbers and also the
> > > > > > > > > steps/programs used the run the same.
> > > > > > > > >
> > > > > > > > > 3) Help us validating the performance test we have done in
> > > > our
> > > > > > test
> > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > close system
> > > > > > calls.
> > > > > > > > >
> > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > >
> > > > > > > > > We used time command to determine how much time the
> > > > > > > > > system took
> > > > to
> > > > > > > > > complete 50000 open/close system calls without
> > > > > > > > > (results attached
> > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > With-auditing-RAW)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > System details:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1 CPU machine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *OS Version*
> > > > > > > > >
> > > > > > > > > RHEL 6.5
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *Kernel Version*
> > > > > > > > >
> > > > > > > > > uname –r
> > > > > > > > >
> > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > for most
> > > > of
> > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks & Regards,
> > > > > > > > >
> > > > > > > > > Logeswari.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Please Donate to
> > > > > > > www.wikipedia.org<http://www.wikipedia.org>
> > > > > >
> > > > > > > --
> > > > > > > Linux-audit mailing list
> > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > >
> > > > > >
> > > > > > - RGB
> > > > > >
> > > > > > --
> > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > Canada
> > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > +1.613.693.0684x3545
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Please Donate to www.wikipedia.org
> > > >
> > > > - RGB
> > > >
> > > > --
> > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > +1.613.693.0684x3545
> > > >
> > >
> > >
> > > --
> > > Please Donate to www.wikipedia.org
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> +1.613.693.0684x3545
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <errno.h>
>
> void create_load(int iters);
> void cleanup();
>
> int high_rate = 0;
> int num_iters = 100000;
> int fd1;
> char file1[50];
> char file2[50];
> char dir1[50];
> char symlink1[50];
>
> /* Purpose: To create system load by invoking system calls used by templates.
> *
> * Note: The unlink(2) of a file can be an expensive operation (i.e., event
> * rate goes way down).
> */
>
> main(int argc, char **argv) {
>
> int num_children=1;
> int iters;
> int i;
> char c;
>
> while ((c = getopt(argc, argv, "hi:")) != -1) {
> switch (c) {
> case 'h':
> /*
> * Desire "high" event rate
> */
> high_rate = 1;
> argc--;
> break;
> case 'i':
> /*
> * Desire a specified number of iterations
> */
> num_iters = atoi(optarg);
> argc--;
> break;
> default:
> fprintf(stderr,"Unknown option: %c\n",optarg);
> exit(1);
> }
> }
>
>
> /*if(argv[optind] != NULL) {
> num_children = atoi(argv[optind]);
> } else {
> num_children = 0;
> }
> Register cleanup routine */
> fprintf(stderr,"Registering cleanup routine...\n");
> if (atexit(cleanup) == -1) {
> fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> errno,strerror(errno));
> exit(1);
> }
>
>
> /* fork child processes, if any requested */
> for(i=1; i < num_children; i++) {
> if(fork() == 0) {
>
> printf("child pid: %d\n",getpid());
>
> /* Setup file names based on child's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> /* each child creates load */
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
> }
> }
>
> /* Parent creates load also */
> printf("parent pid: %d\n",getpid());
>
> /* Setup file names based on parent's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
>
> } /* main */
>
>
> void create_load(int iters) {
>
> int pid;
> char *args[2];
> struct stat stat_buf;
>
> fd1 = creat(file1,0x644);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: creat() returned error for file %s, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> fd1 = open(file1, O_RDWR, 0777);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Chown this file to root instead of user ids so that we don't generate a
> * non-owned alert when the file is truncated when invoking creat() again
> * as root on an existing file owned by another user.
> */
> if (chown(file1,0,0) == -1) {
> fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (fchown(fd1,0,0) == -1) {
> fprintf(stderr,"pid %d: fchown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
>
> if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (ftruncate(fd1,7) == -1) {
> fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> if (truncate(file1,3) == -1) {
> fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rename(file1,file2) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (rename(file2,file1) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file2,file1,errno,strerror(errno));
> exit(1);
> }
> if (link(file1,file2) == -1) {
> fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (symlink(file1,symlink1) == -1) {
> fprintf(stderr,"pid %d: symlink(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,symlink1,errno,strerror(errno));
> exit(1);
> }
> if (lchown(symlink1,0,0) == -1) {
> fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error, errno=%d(%s)\n",
> getpid(),symlink1,0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (lstat(symlink1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (stat(file1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file2) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file2,errno,strerror(errno));
> exit(1);
> }
> if (unlink(symlink1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rmdir(dir1) == -1) {
> fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Fork every 10000 iterations to not use up process resources too quickly */
> if ( (iters % 10000) == 0) {
> pid = fork();
> if(pid == 0) {
> fprintf(stderr,"child pid %d: fork!\n",getpid());
> // child
> args[0] = "/bin/ls";
> args[1] = NULL;
> close(1);
> close(2);
> execve(args[0], args, NULL);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> } else {
> fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
> }
>
> pid = vfork();
> if(pid == 0) {
> args[0] = "/bin/pwd";
> args[1] = NULL;
> close(1);
> close(2);
> execv(args[0], args);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> }
>
> /* Make sure everything is cleaned up and deleted before returning */
> cleanup();
>
> } /* create_load() */
>
> void cleanup() {
> close(fd1);
> unlink(file1);
> unlink(file2);
> unlink(symlink1);
> unlink(dir1);
> return;
> }
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
[-- Attachment #2: audit_log_format.log --]
[-- Type: application/octet-stream, Size: 48259 bytes --]
# ========
# captured on: Thu Feb 12 06:28:47 2015
# hostname : Ostack
# os release : 3.13.11-ckt13
# perf version : 3.13.11-ckt13
# arch : x86_64
# nrcpus online : 24
# nrcpus avail : 24
# cpudesc : Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
# cpuid : GenuineIntel,6,44,2
# total memory : 80414924 kB
# cmdline : /usr/src/linux-3.13.0/tools/perf/perf record -a -F 99
# event : name = cycles, type = 0, config = 0x0, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, excl_host = 0, excl_guest = 1, precise_ip = 0, attr_mmap2 = 0, attr_mmap = 1, attr_mmap_data = 0
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: cpu = 4, software = 1, tracepoint = 2, uncore = 6, breakpoint = 5
# ========
#
# Samples: 3K of event 'cycles'
# Event count (approx.): 102487935703
#
# Overhead Command Shared Object Symbol
# ........ ............. .................. .................................................
#
11.33% loader1 [kernel.kallsyms] [k] format_decode
10.40% loader1 [kernel.kallsyms] [k] memcpy
7.46% loader1 [kernel.kallsyms] [k] number.isra.1
6.99% loader1 [kernel.kallsyms] [k] vsnprintf
2.33% swapper [kernel.kallsyms] [k] intel_idle
1.36% loader1 [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
1.11% loader1 [kernel.kallsyms] [k] ext4_mark_iloc_dirty
1.06% loader1 [kernel.kallsyms] [k] audit_log_vformat
1.02% loader1 [kernel.kallsyms] [k] put_dec_trunc8
0.92% loader1 [kernel.kallsyms] [k] do_get_write_access
0.89% loader1 [kernel.kallsyms] [k] ___ratelimit
0.89% swapper [kernel.kallsyms] [k] ktime_get
0.84% loader1 [kernel.kallsyms] [k] jbd2_journal_put_journal_head
0.84% loader1 [kernel.kallsyms] [k] audit_log_start
0.84% loader1 [kernel.kallsyms] [k] kfree
0.81% loader1 [kernel.kallsyms] [k] jbd2_journal_add_journal_head
0.76% rcu_sched [kernel.kallsyms] [k] native_write_msr_safe
0.74% loader1 [kernel.kallsyms] [k] jbd2_journal_grab_journal_head
0.73% loader1 [kernel.kallsyms] [k] __find_get_block
0.73% loader1 [kernel.kallsyms] [k] __kmalloc_node_track_caller
0.72% loader1 [kernel.kallsyms] [k] __ext4_get_inode_loc
0.72% loader1 [kernel.kallsyms] [k] memset
0.68% loader1 [kernel.kallsyms] [k] _raw_spin_lock
0.67% loader1 [kernel.kallsyms] [k] __alloc_skb
0.67% loader1 [kernel.kallsyms] [k] _raw_spin_lock_irqsave
0.64% perf [kernel.kallsyms] [k] generic_exec_single
0.57% loader1 [kernel.kallsyms] [k] strlen
0.56% loader1 [kernel.kallsyms] [k] __ext4_check_dir_entry
0.54% loader1 [kernel.kallsyms] [k] start_this_handle
0.50% loader1 [kernel.kallsyms] [k] jbd2_journal_dirty_metadata
0.50% loader1 [kernel.kallsyms] [k] audit_log_n_untrustedstring
0.49% loader1 [kernel.kallsyms] [k] _raw_spin_trylock
0.48% mysqld mysqld [.] 0x0000000000591d94
0.48% loader1 [kernel.kallsyms] [k] audit_log_format
0.47% swapper [kernel.kallsyms] [k] apic_timer_interrupt
0.47% loader1 [kernel.kallsyms] [k] kmem_cache_alloc_node
0.46% java [kernel.kallsyms] [k] __d_lookup_rcu
0.45% swapper [kernel.kallsyms] [k] rcu_sysidle_force_exit
0.45% loader1 [kernel.kallsyms] [k] search_dir
0.45% swapper [kernel.kallsyms] [k] int_sqrt
0.45% loader1 [kernel.kallsyms] [k] put_dec_full9
0.45% loader1 [kernel.kallsyms] [k] kmem_cache_free
0.45% swapper [kernel.kallsyms] [k] cpu_startup_entry
0.45% monitor [kernel.kallsyms] [k] int_check_syscall_exit_work
0.44% swapper [kernel.kallsyms] [k] clockevents_program_event
0.44% loader1 [kernel.kallsyms] [k] crc16
0.43% loader1 [kernel.kallsyms] [k] link_path_walk
0.42% loader1 [kernel.kallsyms] [k] __ext4_handle_dirty_metadata
0.42% loader1 [kernel.kallsyms] [k] memcmp
0.41% swapper [kernel.kallsyms] [k] wake_up_process
0.40% loader1 [kernel.kallsyms] [k] skip_atoi
0.40% loader1 [kernel.kallsyms] [k] xattr_resolve_name
0.40% kworker/20:1 [kernel.kallsyms] [k] idle_balance
0.40% loader1 [kernel.kallsyms] [k] skb_release_data
0.39% loader1 [kernel.kallsyms] [k] put_dec
0.39% loader1 [kernel.kallsyms] [k] _cond_resched
0.37% swapper [kernel.kallsyms] [k] perf_adjust_freq_unthr_context.part.79
0.37% swapper [kernel.kallsyms] [k] ktime_get_update_offsets
0.37% loader1 [kernel.kallsyms] [k] audit_log_exit
0.36% loader1 [kernel.kallsyms] [k] jbd2_journal_cancel_revoke
0.34% loader1 [kernel.kallsyms] [k] kmem_cache_alloc
0.34% loader1 [kernel.kallsyms] [k] ext4_find_dest_de
0.34% mysqld libpthread-2.19.so [.] pthread_cond_timedwait@@GLIBC_2.3.2
0.32% loader1 [kernel.kallsyms] [k] __nlmsg_put
0.32% loader1 [kernel.kallsyms] [k] system_call
0.32% loader1 [kernel.kallsyms] [k] unlock_buffer
0.31% loader1 [kernel.kallsyms] [k] __ext4_new_inode
0.31% loader1 [kernel.kallsyms] [k] auditsc_get_stamp
0.30% loader1 [kernel.kallsyms] [k] skb_put
0.29% loader1 [kernel.kallsyms] [k] audit_log_n_string
0.29% loader1 [kernel.kallsyms] [k] audit_log_name
0.29% loader1 [kernel.kallsyms] [k] map_id_up
0.29% loader1 [kernel.kallsyms] [k] strncpy
0.29% loader1 [kernel.kallsyms] [k] audit_log_task_info
0.28% java [kernel.kallsyms] [k] __hrtimer_start_range_ns
0.27% loader1 [kernel.kallsyms] [k] ext4_get_group_desc
0.26% loader1 [kernel.kallsyms] [k] kmem_cache_alloc_trace
0.25% loader1 [kernel.kallsyms] [k] ext4_generic_delete_entry
0.25% sshd [kernel.kallsyms] [k] fput
0.24% loader1 [kernel.kallsyms] [k] bit_waitqueue
0.24% loader1 [kernel.kallsyms] [k] string.isra.5
0.24% loader1 [kernel.kallsyms] [k] prepend_path
0.23% loader1 [kernel.kallsyms] [k] __brelse
0.23% loader1 [kernel.kallsyms] [k] jbd2_journal_stop
0.22% loader1 [kernel.kallsyms] [k] ext4_map_blocks
0.22% loader1 [kernel.kallsyms] [k] audit_filter_inodes
0.22% loader1 [kernel.kallsyms] [k] __audit_inode_child
0.21% loader1 [kernel.kallsyms] [k] audit_log_lost
0.21% loader1 [kernel.kallsyms] [k] __inode_permission
0.21% loader1 [kernel.kallsyms] [k] audit_printk_skb
0.21% loader1 [kernel.kallsyms] [k] kfree_skb
0.21% loader1 [kernel.kallsyms] [k] mutex_lock
0.21% rcuos/5 [kernel.kallsyms] [k] cpumask_next_and
0.19% loader1 [kernel.kallsyms] [k] skb_release_head_state
0.19% loader1 [kernel.kallsyms] [k] down_read
0.19% loader1 [kernel.kallsyms] [k] __audit_syscall_exit
0.19% loader1 [kernel.kallsyms] [k] fsnotify
0.19% loader1 [kernel.kallsyms] [k] strncpy_from_user
0.19% loader1 [kernel.kallsyms] [k] ext4_find_entry
0.18% loader1 [kernel.kallsyms] [k] __ext4_journal_get_write_access
0.18% loader1 [kernel.kallsyms] [k] audit_log_end
0.17% loader1 [kernel.kallsyms] [k] ext4_es_lookup_extent
0.16% loader1 [kernel.kallsyms] [k] from_kgid
0.16% loader1 [kernel.kallsyms] [k] getname_flags
0.16% loader1 [kernel.kallsyms] [k] ext4_reserve_inode_write
0.16% loader1 [kernel.kallsyms] [k] __sb_start_write
0.16% loader1 [kernel.kallsyms] [k] mntput
0.16% loader1 [kernel.kallsyms] [k] strnlen
0.15% loader1 [kernel.kallsyms] [k] mark_page_accessed
0.15% loader1 [kernel.kallsyms] [k] audit_log_untrustedstring
0.14% loader1 [kernel.kallsyms] [k] __audit_inode
0.14% loader1 [kernel.kallsyms] [k] ext4_get_inode_flags
0.14% loader1 [kernel.kallsyms] [k] up_read
0.14% loader1 [kernel.kallsyms] [k] get_vfs_caps_from_disk
0.13% loader1 [kernel.kallsyms] [k] ext4_xattr_ibody_get
0.13% loader1 [kernel.kallsyms] [k] mb_mark_used
0.13% loader1 [kernel.kallsyms] [k] __kmalloc_reserve.isra.26
0.13% loader1 [kernel.kallsyms] [k] mark_buffer_dirty
0.13% loader1 [kernel.kallsyms] [k] dput
0.13% loader1 [kernel.kallsyms] [k] audit_buffer_free
0.13% loader1 [kernel.kallsyms] [k] ext4_xattr_set_handle
0.13% loader1 [kernel.kallsyms] [k] _raw_spin_unlock
0.13% loader1 [kernel.kallsyms] [k] get_page_from_freelist
0.13% loader1 [kernel.kallsyms] [k] sys_close
0.13% loader1 [kernel.kallsyms] [k] skb_free_head
0.11% loader1 [kernel.kallsyms] [k] ksize
0.11% loader1 libc-2.19.so [.] __GI___libc_close
0.11% loader1 [kernel.kallsyms] [k] __d_lookup_rcu
0.11% loader1 [kernel.kallsyms] [k] _raw_read_lock
0.11% loader1 [kernel.kallsyms] [k] ext4_data_block_valid
0.11% loader1 [kernel.kallsyms] [k] jbd2__journal_start
0.11% loader1 [kernel.kallsyms] [k] __printk_ratelimit
0.11% loader1 [kernel.kallsyms] [k] audit_filter_syscall
0.11% loader1 [kernel.kallsyms] [k] path_init
0.11% loader1 [kernel.kallsyms] [k] __mnt_want_write
0.10% loader1 [kernel.kallsyms] [k] lockref_put_or_lock
0.10% loader1 [kernel.kallsyms] [k] prepend_name.isra.9
0.10% loader1 [kernel.kallsyms] [k] __wake_up_bit
0.10% loader1 [kernel.kallsyms] [k] audit_panic
0.10% loader1 [kernel.kallsyms] [k] cap_inode_permission
0.10% loader1 [kernel.kallsyms] [k] ext4_mark_inode_dirty
0.10% loader1 [kernel.kallsyms] [k] notify_change
0.10% loader1 [kernel.kallsyms] [k] __audit_syscall_entry
0.10% loader1 [kernel.kallsyms] [k] ext4_ext_map_blocks
0.10% loader1 [kernel.kallsyms] [k] rb_erase
0.10% loader1 [kernel.kallsyms] [k] __d_lookup
0.10% loader1 [kernel.kallsyms] [k] ext4_free_blocks
0.10% loader1 [kernel.kallsyms] [k] audit_filter_rules.isra.7
0.10% loader1 [kernel.kallsyms] [k] inode_init_always
0.09% loader1 [kernel.kallsyms] [k] complete_walk
0.09% loader1 [kernel.kallsyms] [k] kmalloc_slab
0.09% loader1 [kernel.kallsyms] [k] ima_inode_post_setattr
0.09% loader1 [kernel.kallsyms] [k] wake_up_bit
0.08% loader1 [kernel.kallsyms] [k] do_last
0.08% loader1 [kernel.kallsyms] [k] sys_link
0.08% loader1 [kernel.kallsyms] [k] ext4_read_inode_bitmap
0.08% loader1 [kernel.kallsyms] [k] ext4_evict_inode
0.08% loader1 [kernel.kallsyms] [k] d_walk
0.08% loader1 [kernel.kallsyms] [k] find_next_zero_bit
0.08% loader1 [kernel.kallsyms] [k] audit_copy_inode
0.08% loader1 [kernel.kallsyms] [k] generic_write_sync
0.08% loader1 [kernel.kallsyms] [k] __ext4_ext_check
0.08% loader1 [kernel.kallsyms] [k] __percpu_counter_add
0.08% loader1 [kernel.kallsyms] [k] ext4_discard_preallocations
0.08% loader1 [kernel.kallsyms] [k] filename_lookup
0.08% loader1 [kernel.kallsyms] [k] ext4_inode_table
0.08% loader1 [kernel.kallsyms] [k] __dquot_initialize
0.08% loader1 [kernel.kallsyms] [k] ext4_mb_complex_scan_group
0.08% loader1 [kernel.kallsyms] [k] ext4_getblk
0.08% loader1 [kernel.kallsyms] [k] ima_file_free
0.08% loader1 [kernel.kallsyms] [k] audit_filter_type
0.08% loader1 [kernel.kallsyms] [k] ext4_journal_check_start
0.08% loader1 [kernel.kallsyms] [k] add_dirent_to_buf
0.08% loader1 [kernel.kallsyms] [k] ext4_has_free_clusters
0.08% loader1 [kernel.kallsyms] [k] ext4_release_file
0.08% loader1 [kernel.kallsyms] [k] ext4_setattr
0.08% loader1 [kernel.kallsyms] [k] __block_write_begin
0.08% loader1 [kernel.kallsyms] [k] inode_change_ok
0.08% loader1 [kernel.kallsyms] [k] ext4_has_inline_data
0.08% loader1 [kernel.kallsyms] [k] down_write
0.08% loader1 [kernel.kallsyms] [k] _raw_write_lock
0.08% loader1 [kernel.kallsyms] [k] lockref_get
0.08% loader1 [kernel.kallsyms] [k] common_perm
0.08% loader1 [kernel.kallsyms] [k] setattr_copy
0.08% loader1 [kernel.kallsyms] [k] ext4_truncate
0.08% loader1 [kernel.kallsyms] [k] __es_tree_search
0.07% loader1 [kernel.kallsyms] [k] path_lookupat
0.07% loader1 [kernel.kallsyms] [k] do_unlinkat
0.07% loader1 [kernel.kallsyms] [k] __ext4_journal_start_sb
0.07% loader1 [kernel.kallsyms] [k] ext4_xattr_get
0.07% loader1 [kernel.kallsyms] [k] mb_find_extent
0.07% loader1 [kernel.kallsyms] [k] audit_compare_dname_path
0.07% loader1 [kernel.kallsyms] [k] apparmor_file_open
0.07% loader1 [kernel.kallsyms] [k] mutex_unlock
0.07% loader1 [kernel.kallsyms] [k] skb_release_all
0.07% loader1 [kernel.kallsyms] [k] __getblk
0.06% loader1 [kernel.kallsyms] [k] ext4_group_desc_csum
0.06% loader1 [kernel.kallsyms] [k] __wake_up
0.06% loader1 [kernel.kallsyms] [k] __d_move
0.06% loader1 [kernel.kallsyms] [k] current_kernel_time
0.06% loader1 [kernel.kallsyms] [k] ext4_init_acl
0.06% loader1 [kernel.kallsyms] [k] __generic_file_aio_write
0.06% loader1 [kernel.kallsyms] [k] __alloc_pages_nodemask
0.06% loader1 [kernel.kallsyms] [k] audit_alloc_name
0.06% loader1 [kernel.kallsyms] [k] jbd2_journal_file_inode
0.06% loader1 [kernel.kallsyms] [k] ext4_add_entry
0.06% loader1 [kernel.kallsyms] [k] ____fput
0.06% loader1 [kernel.kallsyms] [k] parent_len
0.06% loader1 [kernel.kallsyms] [k] __call_rcu
0.05% loader1 [kernel.kallsyms] [k] __cleancache_invalidate_inode
0.05% loader1 [kernel.kallsyms] [k] __kmalloc
0.05% loader1 [kernel.kallsyms] [k] __dquot_free_space
0.05% loader1 [kernel.kallsyms] [k] ext4_free_inode
0.05% loader1 [kernel.kallsyms] [k] integrity_iint_find
0.05% loader1 [kernel.kallsyms] [k] empty_dir
0.05% loader1 [kernel.kallsyms] [k] audit_log_d_path
0.05% loader1 [kernel.kallsyms] [k] block_invalidatepage
0.05% loader1 [kernel.kallsyms] [k] jbd2_journal_get_write_access
0.05% loader1 libc-2.19.so [.] __GI___link
0.05% loader1 [kernel.kallsyms] [k] ext4_xattr_set
0.05% loader1 [kernel.kallsyms] [k] audit_serial
0.05% loader1 [kernel.kallsyms] [k] ext4_es_lru_del
0.05% loader1 [kernel.kallsyms] [k] ima_match_policy
0.05% loader1 [kernel.kallsyms] [k] truncate_inode_pages_range
0.05% loader1 [kernel.kallsyms] [k] generic_permission
0.05% loader1 [kernel.kallsyms] [k] __fput
0.05% loader1 [kernel.kallsyms] [k] do_truncate
0.05% loader1 [kernel.kallsyms] [k] find_get_pages
0.05% loader1 [kernel.kallsyms] [k] ext4_mb_free_metadata
0.05% loader1 [kernel.kallsyms] [k] find_get_page
0.05% loader1 [kernel.kallsyms] [k] security_inode_need_killpriv
0.05% loader1 [kernel.kallsyms] [k] user_path_parent
0.05% loader1 [kernel.kallsyms] [k] apparmor_capable
0.05% loader1 [kernel.kallsyms] [k] do_sys_open
0.05% loader1 [kernel.kallsyms] [k] __lookup_hash
0.05% loader1 [kernel.kallsyms] [k] security_task_getsecid
0.05% loader1 [kernel.kallsyms] [k] ext4_rename
0.05% loader1 [kernel.kallsyms] [k] ext4_inode_bitmap
0.05% loader1 [kernel.kallsyms] [k] ima_get_action
0.05% loader1 [kernel.kallsyms] [k] ext4_delete_entry
0.05% loader1 [kernel.kallsyms] [k] handle_dots
0.05% loader1 [kernel.kallsyms] [k] _raw_spin_lock_irq
0.05% loader1 [kernel.kallsyms] [k] jbd2_journal_release_jbd_inode
0.05% loader1 [kernel.kallsyms] [k] vfs_getattr
0.05% loader1 [kernel.kallsyms] [k] generic_getxattr
0.05% loader1 [kernel.kallsyms] [k] strncmp
0.05% loader1 [kernel.kallsyms] [k] cap_inode_getsecid
0.05% loader1 [kernel.kallsyms] [k] add_to_page_cache_lru
0.05% loader1 [kernel.kallsyms] [k] __sb_end_write
0.05% loader1 [kernel.kallsyms] [k] task_tgid_nr_ns
0.05% loader1 [kernel.kallsyms] [k] ext4_mkdir
0.05% loader1 [kernel.kallsyms] [k] vfs_mkdir
0.05% loader1 [kernel.kallsyms] [k] security_inode_permission
0.05% loader1 [kernel.kallsyms] [k] audit_putname
0.05% loader1 [kernel.kallsyms] [k] audit_comparator
0.05% loader1 [kernel.kallsyms] [k] putname
0.05% loader1 [kernel.kallsyms] [k] dquot_drop
0.05% loader1 loader1 [.] create_load
0.05% loader1 [kernel.kallsyms] [k] radix_tree_lookup_element
0.03% loader1 [kernel.kallsyms] [k] mnt_drop_write
0.03% loader1 [kernel.kallsyms] [k] from_kuid
0.03% loader1 [kernel.kallsyms] [k] mem_cgroup_charge_statistics.isra.25
0.03% loader1 [kernel.kallsyms] [k] iput
0.03% loader1 [kernel.kallsyms] [k] set_bh_page
0.03% loader1 [kernel.kallsyms] [k] sys_chown
0.03% loader1 [kernel.kallsyms] [k] d_lru_del
0.03% loader1 [kernel.kallsyms] [k] ext4_claim_free_clusters
0.03% loader1 [kernel.kallsyms] [k] __ext4_journal_stop
0.03% loader1 [kernel.kallsyms] [k] __module_address
0.03% loader1 [kernel.kallsyms] [k] __kmalloc_track_caller
0.03% loader1 [kernel.kallsyms] [k] common_file_perm
0.03% rcu_sched [kernel.kallsyms] [k] autoremove_wake_function
0.03% loader1 [kernel.kallsyms] [k] prepend.constprop.25
0.03% loader1 [kernel.kallsyms] [k] __mark_inode_dirty
0.03% loader1 [kernel.kallsyms] [k] SYSC_newlstat
0.03% loader1 [kernel.kallsyms] [k] ext4_drop_inode
0.03% loader1 [kernel.kallsyms] [k] __mnt_drop_write
0.03% loader1 [kernel.kallsyms] [k] invalidate_inode_buffers
0.03% loader1 [kernel.kallsyms] [k] __inc_zone_state
0.03% loader1 [kernel.kallsyms] [k] ext4_ext_tree_init
0.03% loader1 [kernel.kallsyms] [k] mem_cgroup_uncharge_cache_page
0.03% loader1 [kernel.kallsyms] [k] auditsys
0.03% loader1 [kernel.kallsyms] [k] inode_permission
0.03% loader1 [kernel.kallsyms] [k] insert_inode_locked
0.03% loader1 [kernel.kallsyms] [k] ext4_block_bitmap_csum_set
0.03% loader1 [kernel.kallsyms] [k] radix_tree_insert
0.03% loader1 [kernel.kallsyms] [k] ext4_used_dirs_count
0.03% loader1 [kernel.kallsyms] [k] should_remove_suid
0.03% loader1 [kernel.kallsyms] [k] vfs_unlink
0.03% loader1 [kernel.kallsyms] [k] evm_inode_setattr
0.03% loader1 [kernel.kallsyms] [k] do_notify_resume
0.03% kworker/5:1 [kernel.kallsyms] [k] find_next_bit
0.03% loader1 [kernel.kallsyms] [k] __ext4_read_dirblock
0.03% loader1 [kernel.kallsyms] [k] vfs_truncate
0.03% loader1 loader1 [.] main
0.03% loader1 [kernel.kallsyms] [k] sys_truncate
0.03% loader1 [kernel.kallsyms] [k] ext4_getattr
0.03% loader1 [kernel.kallsyms] [k] ext4_clear_inode
0.03% loader1 [kernel.kallsyms] [k] audit_hold_skb
0.03% loader1 [kernel.kallsyms] [k] generic_file_aio_write
0.03% loader1 [kernel.kallsyms] [k] mntget
0.03% loader1 [kernel.kallsyms] [k] ext4_file_open
0.03% loader1 [kernel.kallsyms] [k] find_lock_page
0.03% loader1 [kernel.kallsyms] [k] ext4_mb_find_by_goal
0.03% rcu_sched [kernel.kallsyms] [k] schedule_timeout
0.03% loader1 [kernel.kallsyms] [k] __find_get_block_slow
0.03% loader1 loader1 [.] chown@plt
0.03% loader1 [kernel.kallsyms] [k] ext4_itable_unused_count
0.03% loader1 [kernel.kallsyms] [k] may_delete
0.03% loader1 [kernel.kallsyms] [k] lookup_page_cgroup
0.03% loader1 [kernel.kallsyms] [k] generic_file_buffered_write
0.03% loader1 [kernel.kallsyms] [k] __zone_watermark_ok
0.03% loader1 libc-2.19.so [.] __GI___symlink
0.03% loader1 [kernel.kallsyms] [k] lookup_dcache
0.03% loader1 [kernel.kallsyms] [k] sys_mkdir
0.03% loader1 [kernel.kallsyms] [k] ext4_orphan_add
0.03% loader1 [kernel.kallsyms] [k] ext4_ext_insert_extent
0.03% loader1 [kernel.kallsyms] [k] apparmor_path_unlink
0.03% loader1 [kernel.kallsyms] [k] lookup_fast
0.03% loader1 libc-2.19.so [.] __GI___mkdir
0.03% loader1 [kernel.kallsyms] [k] free_pages_prepare
0.03% loader1 [kernel.kallsyms] [k] common_perm_create.constprop.23
0.03% loader1 [kernel.kallsyms] [k] __call_rcu_nocb_enqueue
0.03% loader1 [kernel.kallsyms] [k] ext4_da_invalidatepage
0.03% loader1 [kernel.kallsyms] [k] chown_common
0.03% loader1 libc-2.19.so [.] __GI___libc_open
0.03% loader1 [kernel.kallsyms] [k] user_path_at_empty
0.03% rcu_sched [kernel.kallsyms] [k] set_next_entity
0.03% loader1 [kernel.kallsyms] [k] __page_cache_alloc
0.03% loader1 [kernel.kallsyms] [k] ext4_orphan_del
0.03% loader1 [kernel.kallsyms] [k] rcu_note_context_switch
0.03% loader1 [kernel.kallsyms] [k] d_lookup
0.03% loader1 [kernel.kallsyms] [k] __dquot_alloc_space
0.03% loader1 [kernel.kallsyms] [k] ext4_create
0.03% loader1 [kernel.kallsyms] [k] ext4_mb_release_context
0.03% loader1 [kernel.kallsyms] [k] clear_nlink
0.03% loader1 [kernel.kallsyms] [k] cancel_dirty_page
0.03% loader1 [kernel.kallsyms] [k] __cpuset_node_allowed_softwall
0.03% loader1 [kernel.kallsyms] [k] dquot_active.isra.8
0.03% loader1 [kernel.kallsyms] [k] security_file_open
0.03% loader1 [kernel.kallsyms] [k] ext4_link
0.03% rcu_sched [kernel.kallsyms] [k] rb_erase
0.03% loader1 [kernel.kallsyms] [k] ext4_file_write
0.03% loader1 [kernel.kallsyms] [k] process_measurement
0.03% loader1 [kernel.kallsyms] [k] ext4_alloc_inode
0.03% loader1 [kernel.kallsyms] [k] ima_file_check
0.03% loader1 [kernel.kallsyms] [k] sys_creat
0.03% loader1 [kernel.kallsyms] [k] dquot_free_inode
0.02% loader1 [kernel.kallsyms] [k] integrity_inode_free
0.02% loader1 [kernel.kallsyms] [k] ext4_mb_new_blocks
0.02% loader1 [kernel.kallsyms] [k] vfs_rename
0.02% loader1 [kernel.kallsyms] [k] ext4_mb_regular_allocator
0.02% loader1 [kernel.kallsyms] [k] aa_file_perm
0.02% rcu_sched [kernel.kallsyms] [k] force_qs_rnp
0.02% loader1 [kernel.kallsyms] [k] sys_open
0.02% loader1 [kernel.kallsyms] [k] __queue_work
0.02% loader1 [kernel.kallsyms] [k] __wake_up_common
0.02% loader1 [kernel.kallsyms] [k] ext4_unlink
0.02% loader1 [kernel.kallsyms] [k] cpu_needs_another_gp
0.02% loader1 [kernel.kallsyms] [k] common_perm_cond
0.02% loader1 [kernel.kallsyms] [k] apparmor_file_alloc_security
0.02% loader1 [kernel.kallsyms] [k] sys_newlstat
0.02% loader1 [kernel.kallsyms] [k] chmod_common
0.02% loader1 [kernel.kallsyms] [k] ext4_ext_remove_space
0.02% loader1 [kernel.kallsyms] [k] get_empty_filp
0.02% loader1 [kernel.kallsyms] [k] find_group_orlov
0.02% loader1 [kernel.kallsyms] [k] ext4_ext_index_trans_blocks
0.02% loader1 [kernel.kallsyms] [k] put_page
0.02% loader1 [kernel.kallsyms] [k] tick_program_event
0.02% loader1 [kernel.kallsyms] [k] __d_alloc
0.02% loader1 [kernel.kallsyms] [k] d_instantiate
0.02% loader1 [kernel.kallsyms] [k] file_remove_suid
0.02% loader1 [kernel.kallsyms] [k] security_inode_init_security
0.02% kworker/1:0 [kernel.kallsyms] [k] worker_thread
0.02% loader1 [kernel.kallsyms] [k] ext4_acl_chmod
0.02% loader1 [kernel.kallsyms] [k] __srcu_read_lock
0.02% loader1 libc-2.19.so [.] __GI___fchown
0.02% loader1 [kernel.kallsyms] [k] free_hot_cold_page_list
0.02% loader1 [kernel.kallsyms] [k] ext4_es_init_tree
0.02% loader1 [kernel.kallsyms] [k] audit_log_task_context
0.02% loader1 [kernel.kallsyms] [k] zone_statistics
0.02% loader1 [kernel.kallsyms] [k] task_work_run
0.02% loader1 [kernel.kallsyms] [k] path_put
0.02% loader1 [kernel.kallsyms] [k] map_id_down
0.02% loader1 libc-2.19.so [.] __GI___unlink
0.02% loader1 [kernel.kallsyms] [k] d_path
0.02% loader1 [kernel.kallsyms] [k] shrink_dcache_parent
0.02% loader1 [kernel.kallsyms] [k] ext4_get_group_no_and_offset
0.02% loader1 [kernel.kallsyms] [k] ext4_mb_use_preallocated
0.02% loader1 [kernel.kallsyms] [k] vfs_rmdir
0.02% loader1 [kernel.kallsyms] [k] truncate_pagecache
0.02% loader1 [kernel.kallsyms] [k] lockref_get_not_dead
0.02% loader1 [kernel.kallsyms] [k] strcmp
0.02% loader1 [kernel.kallsyms] [k] ext4_mb_unload_buddy.isra.18
0.02% loader1 [kernel.kallsyms] [k] __jbd2_journal_file_buffer
0.02% loader1 [kernel.kallsyms] [k] ext4_inode_attach_jinode
0.02% loader1 [kernel.kallsyms] [k] get_orlov_stats
0.02% loader1 [kernel.kallsyms] [k] fsnotify_clear_marks_by_inode
0.02% loader1 [kernel.kallsyms] [k] apparmor_path_link
0.02% loader1 [kernel.kallsyms] [k] d_splice_alias
0.02% loader1 [kernel.kallsyms] [k] __init_rwsem
0.02% loader1 [kernel.kallsyms] [k] new_inode_pseudo
0.02% loader1 [kernel.kallsyms] [k] list_lru_add
0.02% loader1 [kernel.kallsyms] [k] jbd2_journal_revoke
0.02% loader1 [kernel.kallsyms] [k] ext4_handle_dirty_dirent_node
0.02% loader1 [kernel.kallsyms] [k] user_path_at
0.02% loader1 [kernel.kallsyms] [k] kern_path_create
0.02% loader1 [kernel.kallsyms] [k] cpuacct_charge
0.02% loader1 [kernel.kallsyms] [k] cap_inode_need_killpriv
0.02% loader1 [kernel.kallsyms] [k] ext4_mb_check_limits
0.02% loader1 [kernel.kallsyms] [k] generic_fillattr
0.02% loader1 [kernel.kallsyms] [k] zone_dirty_ok
0.02% loader1 [kernel.kallsyms] [k] inode_has_buffers
0.02% loader1 [kernel.kallsyms] [k] mem_cgroup_page_lruvec
0.02% loader1 [kernel.kallsyms] [k] __ext4_forget
0.02% loader1 [kernel.kallsyms] [k] alloc_pages_current
0.02% loader1 [kernel.kallsyms] [k] __lru_cache_add
0.02% loader1 [kernel.kallsyms] [k] up_write
0.02% loader1 [kernel.kallsyms] [k] set_nlink
0.02% loader1 [kernel.kallsyms] [k] fsnotify_get_cookie
0.02% loader1 [kernel.kallsyms] [k] mb_find_buddy
0.02% loader1 [kernel.kallsyms] [k] ext4_set_inode_state
0.02% loader1 [kernel.kallsyms] [k] alloc_buffer_head
0.02% loader1 [kernel.kallsyms] [k] is_bad_inode
0.02% loader1 [kernel.kallsyms] [k] inode_wb_list_del
0.02% loader1 [kernel.kallsyms] [k] radix_tree_next_chunk
0.02% loader1 [kernel.kallsyms] [k] ext4_dirty_inode
0.02% loader1 [kernel.kallsyms] [k] __dentry_kill
0.02% loader1 [kernel.kallsyms] [k] security_inode_getattr
0.02% loader1 [kernel.kallsyms] [k] ext4_find_delalloc_range
0.02% loader1 [kernel.kallsyms] [k] pagevec_lookup
0.02% loader1 [kernel.kallsyms] [k] set_next_entity
0.02% loader1 [kernel.kallsyms] [k] int_restore_rest
0.02% loader1 [kernel.kallsyms] [k] inode_init_once
0.02% loader1 [kernel.kallsyms] [k] hrtimer_run_queues
0.02% loader1 [kernel.kallsyms] [k] final_putname
0.02% loader1 libc-2.19.so [.] __GI___libc_write
0.02% loader1 [kernel.kallsyms] [k] __mem_cgroup_commit_charge
0.02% loader1 [kernel.kallsyms] [k] cap_inode_rename
0.02% loader1 [kernel.kallsyms] [k] sys_fchown
0.02% loader1 [kernel.kallsyms] [k] audit_log_key
0.02% loader1 [kernel.kallsyms] [k] ext4_bread
0.02% loader1 [kernel.kallsyms] [k] ext4_add_nondir
0.02% loader1 [kernel.kallsyms] [k] wait_for_stable_page
0.02% loader1 [kernel.kallsyms] [k] ext4_es_free_extent
0.02% loader1 [kernel.kallsyms] [k] SYSC_renameat
0.02% loader1 [kernel.kallsyms] [k] copy_user_generic_string
0.02% loader1 [kernel.kallsyms] [k] security_inode_setattr
0.02% loader1 [kernel.kallsyms] [k] __alloc_fd
0.02% loader1 [kernel.kallsyms] [k] __inc_zone_page_state
0.02% loader1 [kernel.kallsyms] [k] ll_rw_block
0.02% loader1 [kernel.kallsyms] [k] next_zones_zonelist
0.02% kworker/7:1 [kernel.kallsyms] [k] idle_cpu
0.02% ovs-vswitchd [openvswitch] [k] ovs_flow_tbl_dump_next
0.02% swapper [kernel.kallsyms] [k] _raw_spin_lock
0.02% swapper [kernel.kallsyms] [k] menu_select
0.02% java [kernel.kallsyms] [k] find_next_bit
0.02% rcuos/1 [kernel.kallsyms] [k] default_send_IPI_mask_sequence_phys
0.02% rcuos/1 [kernel.kallsyms] [k] pick_next_task_stop
0.01% swapper [kernel.kallsyms] [k] rcu_eqs_enter_common.isra.48
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] find_get_page
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] __slab_free
0.01% swapper [kernel.kallsyms] [k] tick_nohz_restart
0.01% kworker/u66:2 [kernel.kallsyms] [k] ext4_ext_try_to_merge
0.01% swapper [kernel.kallsyms] [k] __remove_hrtimer
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] jbd2_clear_buffer_revoked_flags
0.01% java libpthread-2.19.so [.] pthread_cond_timedwait@@GLIBC_2.3.2
0.01% swapper [kernel.kallsyms] [k] native_write_msr_safe
0.01% swapper [kernel.kallsyms] [k] rcu_idle_enter
0.01% swapper [kernel.kallsyms] [k] __schedule
0.01% ovs-vswitchd [kernel.kallsyms] [k] netlink_rcv_wake
0.01% java [kernel.kallsyms] [k] sysret_audit
0.01% java [kernel.kallsyms] [k] __schedule
0.01% swapper [kernel.kallsyms] [k] rcu_eqs_exit_common.isra.49
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] put_page
0.01% swapper [kernel.kallsyms] [k] read_tsc
0.01% swapper [kernel.kallsyms] [k] notifier_call_chain
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] __rmqueue
0.01% swapper [kernel.kallsyms] [k] rcu_sysidle_enter
0.00% swapper [kernel.kallsyms] [k] run_posix_cpu_timers
0.00% java libjvm.so [.] _ZN25ContiguousSpaceUsedHelper11take_sampleEv
0.00% jbd2/sdb1-8 [kernel.kallsyms] [k] __find_get_block_slow
0.00% java [kernel.kallsyms] [k] load_balance
0.00% perf [kernel.kallsyms] [k] __perf_event_enable
0.00% loader1 [kernel.kallsyms] [k] native_write_msr_safe
0.00% perf [kernel.kallsyms] [k] native_write_msr_safe
#
# (For a higher level overview, try: perf report --sort comm,dso)
#
[-- Attachment #3: audit_log_format_cg.log --]
[-- Type: application/octet-stream, Size: 621916 bytes --]
# ========
# captured on: Thu Feb 12 06:19:23 2015
# hostname : Ostack
# os release : 3.13.11-ckt13
# perf version : 3.13.11-ckt13
# arch : x86_64
# nrcpus online : 24
# nrcpus avail : 24
# cpudesc : Intel(R) Xeon(R) CPU E5649 @ 2.53GHz
# cpuid : GenuineIntel,6,44,2
# total memory : 80414924 kB
# cmdline : /usr/src/linux-3.13.0/tools/perf/perf record -a -F 99 -g
# event : name = cycles, type = 0, config = 0x0, config1 = 0x0, config2 = 0x0, excl_usr = 0, excl_kern = 0, excl_host = 0, excl_guest = 1, precise_ip = 0, attr_mmap2 = 0, attr_mmap = 1, attr_mmap_data = 0
# HEADER_CPU_TOPOLOGY info available, use -I to display
# HEADER_NUMA_TOPOLOGY info available, use -I to display
# pmu mappings: cpu = 4, software = 1, tracepoint = 2, uncore = 6, breakpoint = 5
# ========
#
# Samples: 2K of event 'cycles'
# Event count (approx.): 58048445085
#
# Overhead Command Shared Object Symbol
# ........ ............. ................. .......................................
#
11.85% loader1 [kernel.kallsyms] [k] format_decode
|
--- format_decode
|
|--89.02%-- vsnprintf
| audit_log_vformat
| audit_log_format
| |
| |--35.84%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--23.39%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--19.41%-- __GI___unlink
| | | |
| | | |--67.23%-- main
| | | | __libc_start_main
| | | |
| | | --32.77%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--10.04%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--7.71%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--7.35%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--5.20%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | |--5.20%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--4.68%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--3.50%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--3.50%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--2.69%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--2.51%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--2.33%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--1.34%-- truncate
| | | main
| | | __libc_start_main
| | |
| | --1.16%-- __GI___mkdir
| | main
| | __libc_start_main
| |
| |--28.44%-- audit_log_start
| | |
| | |--65.65%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--26.32%-- __GI___unlink
| | | | |
| | | | |--74.51%-- create_load
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --25.49%-- main
| | | | __libc_start_main
| | | |
| | | |--9.99%-- rename
| | | | main
| | | | __libc_start_main
| | | |
| | | |--7.74%-- __GI___libc_close
| | | | |
| | | | |--66.67%-- create_load
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --33.33%-- main
| | | | __libc_start_main
| | | |
| | | |--7.30%-- __GI___fchown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--7.05%-- __lxstat64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.82%-- __xstat64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.82%-- __GI___libc_open
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.78%-- __GI___libc_chown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.48%-- __GI___mkdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.47%-- __GI___ftruncate64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.47%-- __GI___lchown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.58%-- __creat_nocancel
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.24%-- truncate
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.24%-- __GI___symlink
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.24%-- __GI___chmod
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.24%-- __GI___link
| | | | main
| | | | __libc_start_main
| | | |
| | | --2.24%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | --34.35%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--27.02%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--23.33%-- __GI___unlink
| | | |
| | | |--63.36%-- main
| | | | __libc_start_main
| | | |
| | | --36.64%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--14.13%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--9.21%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--8.55%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--4.93%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--4.27%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--4.27%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | --4.27%-- __creat_nocancel
| | main
| | __libc_start_main
| |
| |--17.33%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--33.17%-- __GI___unlink
| | | |
| | | |--53.09%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --46.91%-- main
| | | __libc_start_main
| | |
| | |--17.62%-- __GI___libc_close
| | | |
| | | |--58.89%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --41.11%-- main
| | | __libc_start_main
| | |
| | |--9.64%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--5.29%-- truncate
| | | main
| | | __libc_start_main
| | |
| | |--5.19%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | |--2.78%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--2.78%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--2.78%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--2.78%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--2.78%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--2.78%-- __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | |--2.77%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | |--2.41%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--2.41%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--2.41%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | --2.41%-- rename
| | main
| | __libc_start_main
| |
| |--16.12%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--19.71%-- __GI___libc_close
| | | |
| | | |--71.69%-- main
| | | | __libc_start_main
| | | |
| | | --28.31%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--18.93%-- __GI___unlink
| | | |
| | | |--72.64%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --27.36%-- main
| | | __libc_start_main
| | |
| | |--7.77%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--5.98%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--5.98%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--5.58%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--5.36%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--5.18%-- __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | |--5.18%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--5.18%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | |--4.92%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--2.61%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--2.59%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--2.59%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | --2.43%-- __GI___lchown
| | main
| | __libc_start_main
| |
| --2.28%-- audit_log_key
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--21.15%-- __GI___fchmod
| | main
| | __libc_start_main
| |
| |--21.12%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--21.11%-- __creat_nocancel
| | main
| | __libc_start_main
| |
| |--18.31%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --18.31%-- __GI___unlink
| create_load
| main
| __libc_start_main
|
--10.98%-- audit_log_vformat
audit_log_format
|
|--38.80%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--18.80%-- __GI___unlink
| | main
| | __libc_start_main
| |
| |--18.79%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| |--17.45%-- __GI___symlink
| | main
| | __libc_start_main
| |
| |--10.06%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--8.72%-- __GI___lchown
| | main
| | __libc_start_main
| |
| |--8.72%-- __GI___fchown
| | main
| | __libc_start_main
| |
| |--8.72%-- __xstat64
| | main
| | __libc_start_main
| |
| --8.72%-- __GI___chmod
| main
| __libc_start_main
|
|--29.70%-- audit_log_start
| |
| |--86.87%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--41.49%-- __GI___unlink
| | | |
| | | |--68.08%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --31.92%-- main
| | | __libc_start_main
| | |
| | |--28.24%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--15.13%-- __GI___libc_close
| | | main
| | | __libc_start_main
| | |
| | --15.13%-- __GI___libc_write
| | main
| | __libc_start_main
| |
| --13.13%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| truncate
| main
| __libc_start_main
|
|--17.44%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--22.38%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| |--19.40%-- __GI___ftruncate64
| | main
| | __libc_start_main
| |
| |--19.40%-- __GI___unlink
| | main
| | __libc_start_main
| |
| |--19.40%-- __GI___chmod
| | main
| | __libc_start_main
| |
| --19.40%-- truncate
| main
| __libc_start_main
|
--14.06%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--27.76%-- rename
| main
| __libc_start_main
|
|--24.08%-- __lxstat64
| main
| __libc_start_main
|
|--24.08%-- __xstat64
| main
| __libc_start_main
|
--24.08%-- __GI___unlink
main
__libc_start_main
9.99% loader1 [kernel.kallsyms] [k] memcpy
|
--- memcpy
|
|--94.81%-- audit_log_vformat
| audit_log_format
| |
| |--32.38%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--33.58%-- __GI___unlink
| | | |
| | | |--64.53%-- main
| | | | __libc_start_main
| | | |
| | | --35.47%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--21.50%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--11.93%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--5.82%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--4.53%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | |--4.53%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--3.31%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--3.09%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--3.08%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--2.87%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--2.87%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--1.44%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | --1.44%-- __GI___fchmod
| | main
| | __libc_start_main
| |
| |--32.37%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--16.25%-- __GI___unlink
| | | |
| | | |--55.77%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --44.23%-- main
| | | __libc_start_main
| | |
| | |--14.81%-- __GI___libc_close
| | | |
| | | |--61.19%-- main
| | | | __libc_start_main
| | | |
| | | --38.81%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--8.84%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--7.85%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--6.30%-- __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | |--6.18%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--5.98%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--4.75%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--4.75%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--3.10%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--3.09%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--3.09%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--3.09%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | |--2.87%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--2.87%-- truncate
| | | main
| | | __libc_start_main
| | |
| | |--1.66%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--1.64%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--1.44%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | --1.44%-- __GI___lchown
| | main
| | __libc_start_main
| |
| |--17.47%-- audit_log_start
| | |
| | |--66.60%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--26.19%-- __GI___unlink
| | | | |
| | | | |--67.12%-- create_load
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --32.88%-- main
| | | | __libc_start_main
| | | |
| | | |--8.61%-- rename
| | | | main
| | | | __libc_start_main
| | | |
| | | |--8.00%-- __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--8.00%-- __GI___ftruncate64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--8.00%-- __lxstat64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--8.00%-- __GI___libc_chown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.61%-- __xstat64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.61%-- __GI___libc_write
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.00%-- truncate
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.00%-- __GI___fchown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.00%-- __GI___mkdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.00%-- __GI___link
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.00%-- __creat_nocancel
| | | | main
| | | | __libc_start_main
| | | |
| | | --3.99%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | --33.40%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--65.70%-- __GI___unlink
| | | |
| | | |--75.73%-- main
| | | | __libc_start_main
| | | |
| | | --24.27%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--9.20%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--9.16%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--7.97%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | --7.97%-- rename
| | main
| | __libc_start_main
| |
| |--16.78%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--17.48%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--17.06%-- __GI___unlink
| | | |
| | | |--65.00%-- main
| | | | __libc_start_main
| | | |
| | | --35.00%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--11.51%-- __GI___libc_close
| | | |
| | | |--75.92%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --24.08%-- main
| | | __libc_start_main
| | |
| | |--8.74%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--6.40%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--5.97%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--5.97%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--5.97%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | |--5.76%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--3.20%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--3.20%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | |--3.20%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--2.77%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | --2.77%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --1.00%-- audit_log_key
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.57%-- __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| --46.43%-- __GI___fchmod
| main
| __libc_start_main
|
--5.19%-- audit_log_n_untrustedstring
audit_log_untrustedstring
|
|--71.89%-- audit_log_d_path
| |
| |--50.90%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--76.77%-- __GI___unlink
| | | |
| | | |--65.14%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --34.86%-- main
| | | __libc_start_main
| | |
| | --23.23%-- __GI___libc_chown
| | main
| | __libc_start_main
| |
| --49.10%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--72.23%-- __GI___unlink
| | |
| | |--66.67%-- create_load
| | | main
| | | __libc_start_main
| | |
| | --33.33%-- main
| | __libc_start_main
| |
| --27.77%-- rename
| main
| __libc_start_main
|
|--18.31%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.57%-- __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| --46.43%-- __GI___ftruncate64
| main
| __libc_start_main
|
--9.81%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
truncate
main
__libc_start_main
7.85% loader1 [kernel.kallsyms] [k] number.isra.1
|
--- number.isra.1
|
|--96.98%-- vsnprintf
| audit_log_vformat
| audit_log_format
| |
| |--35.39%-- audit_log_start
| | |
| | |--66.68%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--28.08%-- __GI___unlink
| | | | |
| | | | |--73.84%-- create_load
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --26.16%-- main
| | | | __libc_start_main
| | | |
| | | |--13.29%-- rename
| | | | main
| | | | __libc_start_main
| | | |
| | | |--7.35%-- __GI___link
| | | | main
| | | | __libc_start_main
| | | |
| | | |--5.59%-- __GI___symlink
| | | | main
| | | | __libc_start_main
| | | |
| | | |--5.28%-- __xstat64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--5.28%-- __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.96%-- __GI___mkdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.90%-- __lxstat64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.90%-- truncate
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.90%-- __GI___fchmod
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.83%-- __GI___libc_close
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.83%-- __creat_nocancel
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.45%-- __GI___fchown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.45%-- __GI___libc_open
| | | | main
| | | | __libc_start_main
| | | |
| | | |--2.45%-- __GI___chmod
| | | | main
| | | | __libc_start_main
| | | |
| | | --2.45%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | --33.32%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--31.70%-- __GI___unlink
| | | |
| | | |--66.67%-- main
| | | | __libc_start_main
| | | |
| | | --33.33%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--20.38%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--16.23%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--9.81%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | |--5.66%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--5.66%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--5.65%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | --4.91%-- __creat_nocancel
| | main
| | __libc_start_main
| |
| |--26.88%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--28.97%-- __GI___unlink
| | | |
| | | |--68.00%-- main
| | | | __libc_start_main
| | | |
| | | --32.00%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--22.21%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--13.58%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--7.32%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--4.96%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--4.64%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | |--4.31%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--2.48%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--2.48%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--2.48%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--2.26%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--2.15%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | --2.15%-- __lxstat64
| | main
| | __libc_start_main
| |
| |--19.75%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--28.16%-- __GI___unlink
| | | |
| | | |--55.20%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --44.80%-- main
| | | __libc_start_main
| | |
| | |--15.09%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--12.17%-- __GI___libc_close
| | | |
| | | |--75.93%-- main
| | | | __libc_start_main
| | | |
| | | --24.07%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--6.72%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--6.31%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--6.31%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--6.28%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--3.38%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--3.38%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--3.37%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--2.96%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--2.93%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | --2.93%-- __lxstat64
| | main
| | __libc_start_main
| |
| --17.98%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--26.70%-- __GI___unlink
| | |
| | |--74.16%-- create_load
| | | main
| | | __libc_start_main
| | |
| | --25.84%-- main
| | __libc_start_main
| |
| |--13.86%-- __GI___libc_close
| | |
| | |--76.78%-- main
| | | __libc_start_main
| | |
| | --23.22%-- create_load
| | main
| | __libc_start_main
| |
| |--10.63%-- __GI___symlink
| | main
| | __libc_start_main
| |
| |--6.93%-- __GI___mkdir
| | main
| | __libc_start_main
| |
| |--6.93%-- rename
| | main
| | __libc_start_main
| |
| |--6.92%-- __creat_nocancel
| | main
| | __libc_start_main
| |
| |--6.44%-- __GI___link
| | main
| | __libc_start_main
| |
| |--3.71%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--3.71%-- __xstat64
| | main
| | __libc_start_main
| |
| |--3.71%-- __GI___chmod
| | main
| | __libc_start_main
| |
| |--3.63%-- __GI___fchmod
| | main
| | __libc_start_main
| |
| |--3.45%-- __GI___libc_chown
| | main
| | __libc_start_main
| |
| --3.38%-- __GI___libc_open
| main
| __libc_start_main
|
--3.02%-- audit_log_vformat
audit_log_format
|
|--39.98%-- audit_log_start
| audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.56%-- truncate
| | main
| | __libc_start_main
| |
| --46.44%-- __GI___unlink
| main
| __libc_start_main
|
|--38.59%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--51.89%-- __lxstat64
| | main
| | __libc_start_main
| |
| --48.11%-- truncate
| main
| __libc_start_main
|
--21.42%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___libc_write
main
__libc_start_main
6.00% loader1 [kernel.kallsyms] [k] vsnprintf
|
--- vsnprintf
|
|--97.80%-- audit_log_vformat
| audit_log_format
| |
| |--28.93%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--43.42%-- __GI___unlink
| | | |
| | | |--58.63%-- main
| | | | __libc_start_main
| | | |
| | | --41.37%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--11.66%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--8.59%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--8.37%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--8.19%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--5.99%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--3.00%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--3.00%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | |--2.60%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--2.60%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | --2.60%-- __GI___symlink
| | main
| | __libc_start_main
| |
| |--26.83%-- audit_log_start
| | |
| | |--70.65%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--16.47%-- __creat_nocancel
| | | | main
| | | | __libc_start_main
| | | |
| | | |--13.11%-- rename
| | | | main
| | | | __libc_start_main
| | | |
| | | |--13.11%-- __GI___link
| | | | main
| | | | __libc_start_main
| | | |
| | | |--12.50%-- __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--7.93%-- __GI___ftruncate64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--7.93%-- __GI___unlink
| | | | create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.57%-- __lxstat64
| | | | main
| | | | __libc_start_main
| | | |
| | | |--4.57%-- __GI___libc_open
| | | | main
| | | | __libc_start_main
| | | |
| | | |--3.96%-- __GI___fchown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--3.96%-- truncate
| | | | main
| | | | __libc_start_main
| | | |
| | | |--3.96%-- __GI___fchmod
| | | | main
| | | | __libc_start_main
| | | |
| | | |--3.96%-- __GI___mkdir
| | | | main
| | | | __libc_start_main
| | | |
| | | --3.96%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | --29.35%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--31.75%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--28.62%-- __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | |--19.08%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--11.00%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | --9.55%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| |--25.86%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--37.10%-- __GI___unlink
| | | |
| | | |--84.32%-- create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --15.68%-- main
| | | __libc_start_main
| | |
| | |--12.51%-- __GI___libc_close
| | | |
| | | |--76.78%-- main
| | | | __libc_start_main
| | | |
| | | --23.22%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--9.61%-- truncate
| | | main
| | | __libc_start_main
| | |
| | |--6.26%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--6.26%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--6.26%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--3.35%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--3.35%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--3.35%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--3.23%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--2.91%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--2.91%-- rename
| | | main
| | | __libc_start_main
| | |
| | --2.91%-- __GI___lchown
| | main
| | __libc_start_main
| |
| |--14.33%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--32.26%-- __GI___unlink
| | | |
| | | |--51.25%-- main
| | | | __libc_start_main
| | | |
| | | --48.75%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--17.34%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--12.10%-- __GI___libc_close
| | | main
| | | __libc_start_main
| | |
| | |--6.05%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--6.05%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--5.24%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--5.24%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--5.24%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | |--5.24%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | --5.24%-- __GI___chmod
| | main
| | __libc_start_main
| |
| |--2.55%-- audit_log_d_path
| | |
| | |--67.91%-- audit_log_task_info
| | | audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--50.01%-- rename
| | | | main
| | | | __libc_start_main
| | | |
| | | --49.99%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | --32.09%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___chmod
| | main
| | __libc_start_main
| |
| --1.50%-- audit_log_key
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--50.00%-- rename
| | main
| | __libc_start_main
| |
| --50.00%-- __xstat64
| main
| __libc_start_main
|
--2.20%-- audit_log_format
|
|--66.67%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| rename
| main
| __libc_start_main
|
--33.33%-- audit_log_key
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___chmod
main
__libc_start_main
4.24% swapper [kernel.kallsyms] [k] intel_idle
|
--- intel_idle
cpuidle_enter_state
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
|
|--85.41%-- start_secondary
|
--14.59%-- rest_init
start_kernel
x86_64_start_reservations
x86_64_start_kernel
1.66% loader1 [kernel.kallsyms] [k] down_read
|
--- down_read
|
|--91.65%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--93.77%-- __GI___mkdir
| | main
| | __libc_start_main
| |
| |--3.34%-- __GI___libc_write
| | main
| | __libc_start_main
| |
| --2.89%-- __xstat64
| main
| __libc_start_main
|
|--5.70%-- ext4_xattr_get
| ext4_xattr_security_get
| generic_getxattr
| get_vfs_caps_from_disk
| audit_copy_inode
| |
| |--53.55%-- __audit_inode_child
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --46.45%-- __audit_inode
| filename_lookup
| kern_path_create
| user_path_create
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
--2.65%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___rmdir
main
__libc_start_main
1.27% loader1 [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
|
--- _raw_spin_unlock_irqrestore
|
|--47.20%-- ___ratelimit
| __printk_ratelimit
| |
| |--45.19%-- audit_log_lost
| | audit_printk_skb
| | audit_log_end
| | |
| | |--65.03%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--50.00%-- __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--25.00%-- __creat_nocancel
| | | | main
| | | | __libc_start_main
| | | |
| | | --25.00%-- rename
| | | main
| | | __libc_start_main
| | |
| | --34.97%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--53.51%-- rename
| | | main
| | | __libc_start_main
| | |
| | --46.49%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--31.65%-- audit_printk_skb
| | audit_log_end
| | |
| | |--50.01%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--53.58%-- __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | --46.42%-- __GI___libc_close
| | | main
| | | __libc_start_main
| | |
| | --49.99%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--53.56%-- rename
| | | main
| | | __libc_start_main
| | |
| | --46.44%-- __GI___lchown
| | main
| | __libc_start_main
| |
| --23.16%-- audit_panic
| audit_log_lost
| audit_printk_skb
| audit_log_end
| |
| |--68.28%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___unlink
| | |
| | |--53.55%-- create_load
| | | main
| | | __libc_start_main
| | |
| | --46.45%-- main
| | __libc_start_main
| |
| --31.72%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| create_load
| main
| __libc_start_main
|
|--18.93%-- audit_buffer_free
| audit_log_end
| |
| |--57.78%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--36.58%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--31.71%-- __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | --31.71%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --42.22%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___libc_chown
| main
| __libc_start_main
|
|--18.40%-- audit_log_start
| |
| |--56.54%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--33.33%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | |--33.33%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | --33.33%-- __GI___libc_close
| | main
| | __libc_start_main
| |
| --43.46%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--50.01%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --49.99%-- __lxstat64
| main
| __libc_start_main
|
|--4.00%-- audit_serial
| auditsc_get_stamp
| audit_log_start
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___libc_open
| main
| __libc_start_main
|
|--4.00%-- __printk_ratelimit
| audit_printk_skb
| audit_log_end
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| truncate
| main
| __libc_start_main
|
|--4.00%-- __wake_up
| jbd2_journal_stop
| __ext4_journal_stop
| ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--3.47%-- audit_log_end
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___libc_chown
main
__libc_start_main
1.24% loader1 [kernel.kallsyms] [k] ext4_mark_iloc_dirty
|
--- ext4_mark_iloc_dirty
|
|--89.36%-- ext4_mark_inode_dirty
| |
| |--21.04%-- add_dirent_to_buf
| | ext4_add_entry
| | |
| | |--37.63%-- ext4_rename
| | | vfs_rename
| | | SYSC_renameat
| | | sys_rename
| | | system_call
| | | rename
| | | main
| | | __libc_start_main
| | |
| | |--21.76%-- ext4_mkdir
| | | vfs_mkdir
| | | sys_mkdir
| | | system_call
| | | __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--21.75%-- ext4_link
| | | vfs_link
| | | sys_link
| | | system_call
| | | __GI___link
| | | main
| | | __libc_start_main
| | |
| | --18.85%-- ext4_add_nondir
| | ext4_create
| | vfs_create
| | do_last
| | path_openat
| | do_filp_open
| | do_sys_open
| | sys_creat
| | system_call
| | __creat_nocancel
| | main
| | __libc_start_main
| |
| |--15.87%-- ext4_dirty_inode
| | __mark_inode_dirty
| | |
| | |--50.00%-- ext4_setattr
| | | notify_change
| | | |
| | | |--50.00%-- chmod_common
| | | | sys_chmod
| | | | system_call
| | | | __GI___chmod
| | | | main
| | | | __libc_start_main
| | | |
| | | --50.00%-- chown_common
| | | sys_chown
| | | system_call
| | | __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--25.00%-- generic_write_end
| | | ext4_da_write_end
| | | generic_file_buffered_write
| | | __generic_file_aio_write
| | | generic_file_aio_write
| | | ext4_file_write
| | | do_sync_write
| | | vfs_write
| | | sys_write
| | | system_call
| | | __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | --25.00%-- ext4_mb_new_blocks
| | ext4_ext_map_blocks
| | ext4_map_blocks
| | ext4_getblk
| | ext4_bread
| | ext4_append
| | ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| |--13.06%-- ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| |--11.90%-- ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| |--9.15%-- __ext4_new_inode
| | ext4_symlink
| | vfs_symlink
| | sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| |--8.54%-- ext4_ext_tree_init
| | __ext4_new_inode
| | |
| | |--53.56%-- ext4_symlink
| | | vfs_symlink
| | | sys_symlink
| | | system_call
| | | __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | --46.44%-- ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| |--7.93%-- ext4_evict_inode
| | evict
| | iput
| | d_delete
| | vfs_rmdir
| | do_rmdir
| | sys_rmdir
| | system_call
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| |--4.58%-- ext4_add_nondir
| | ext4_symlink
| | vfs_symlink
| | sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| |--3.97%-- ext4_truncate
| | ext4_setattr
| | notify_change
| | do_truncate
| | vfs_truncate
| | do_sys_truncate
| | sys_truncate
| | system_call
| | truncate
| | main
| | __libc_start_main
| |
| --3.97%-- ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
|--7.09%-- ext4_orphan_add
| ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--3.55%-- ext4_orphan_del
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
1.20% loader1 [kernel.kallsyms] [k] audit_log_vformat
|
--- audit_log_vformat
|
|--92.67%-- audit_log_format
| |
| |--28.97%-- audit_log_start
| | |
| | |--70.60%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--22.32%-- rename
| | | | main
| | | | __libc_start_main
| | | |
| | | |--19.62%-- __GI___libc_open
| | | | main
| | | | __libc_start_main
| | | |
| | | |--19.39%-- truncate
| | | | main
| | | | __libc_start_main
| | | |
| | | |--19.34%-- __GI___link
| | | | main
| | | | __libc_start_main
| | | |
| | | --19.34%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | --29.40%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___unlink
| | main
| | __libc_start_main
| |
| |--20.52%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--41.55%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--19.89%-- __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | |--19.28%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | --19.28%-- rename
| | main
| | __libc_start_main
| |
| |--17.04%-- audit_log_key
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--26.79%-- __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | |--26.78%-- __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | |--23.21%-- rename
| | | main
| | | __libc_start_main
| | |
| | --23.21%-- __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| |--13.08%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--34.89%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--34.87%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | --30.24%-- __GI___chmod
| | main
| | __libc_start_main
| |
| |--12.48%-- audit_log_d_path
| | |
| | |--68.30%-- audit_log_task_info
| | | audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | --31.70%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| --7.91%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--50.00%-- __GI___libc_close
| | main
| | __libc_start_main
| |
| --50.00%-- __xstat64
| main
| __libc_start_main
|
|--3.67%-- audit_log_start
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___ftruncate64
| main
| __libc_start_main
|
--3.67%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
1.17% swapper [kernel.kallsyms] [k] menu_select
|
--- menu_select
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
|
|--89.26%-- start_secondary
|
--10.74%-- rest_init
start_kernel
x86_64_start_reservations
x86_64_start_kernel
1.01% rcu_sched [kernel.kallsyms] [k] native_write_msr_safe
|
--- native_write_msr_safe
0.95% loader1 [kernel.kallsyms] [k] put_dec_trunc8
|
--- put_dec_trunc8
|
|--94.65%-- put_dec
| number.isra.1
| vsnprintf
| audit_log_vformat
| audit_log_format
| |
| |--69.12%-- audit_log_start
| | |
| | |--62.13%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--39.47%-- __GI___libc_open
| | | | main
| | | | __libc_start_main
| | | |
| | | |--13.16%-- __GI___unlink
| | | | main
| | | | __libc_start_main
| | | |
| | | |--13.16%-- __GI___libc_close
| | | | main
| | | | __libc_start_main
| | | |
| | | |--11.40%-- __GI___fchmod
| | | | main
| | | | __libc_start_main
| | | |
| | | |--11.40%-- __GI___fchown
| | | | main
| | | | __libc_start_main
| | | |
| | | --11.40%-- __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | --37.87%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--56.99%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--21.59%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | --21.42%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| |--25.98%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--37.70%-- __GI___unlink
| | | |
| | | |--50.00%-- main
| | | | __libc_start_main
| | | |
| | | --50.00%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--21.74%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--21.71%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | --18.85%-- truncate
| | main
| | __libc_start_main
| |
| --4.90%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| truncate
| main
| __libc_start_main
|
--5.35%-- number.isra.1
vsnprintf
audit_log_vformat
audit_log_format
audit_log_start
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
create_load
main
__libc_start_main
0.91% loader1 [kernel.kallsyms] [k] _raw_spin_lock
|
--- _raw_spin_lock
|
|--10.42%-- iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--9.93%-- lockref_put_or_lock
| dput
| |
| |--51.28%-- path_put
| | vfs_fstatat
| | SYSC_newlstat
| | sys_newlstat
| | system_call
| | __lxstat64
| | main
| | __libc_start_main
| |
| --48.72%-- __fput
| ____fput
| task_work_run
| do_notify_resume
| int_signal
| __GI___libc_close
| main
| __libc_start_main
|
|--9.68%-- __audit_getname
| getname_flags
| |
| |--50.00%-- user_path_parent
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| --50.00%-- user_path_at_empty
| user_path_at
| sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
|--5.58%-- jbd2_journal_begin_ordered_truncate
| ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
|--5.58%-- get_task_comm
| audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| main
| __libc_start_main
|
|--5.58%-- new_inode_pseudo
| new_inode
| __ext4_new_inode
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
|--4.85%-- __find_get_block
| __getblk
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
|--4.84%-- __mark_inode_dirty
| ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
|--4.84%-- d_alloc
| lookup_dcache
| __lookup_hash
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--4.84%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--4.84%-- ext4_inode_attach_jinode
| ext4_file_open
| do_dentry_open
| vfs_open
| do_last
| path_openat
| do_filp_open
| do_sys_open
| sys_creat
| system_call
| __creat_nocancel
| main
| __libc_start_main
|
|--4.84%-- ext4_clear_inode
| ext4_free_inode
| ext4_evict_inode
| evict
| iput
| d_delete
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
|--4.84%-- __remove_inode_hash
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--4.84%-- res_counter_uncharge_until
| res_counter_uncharge
| mem_cgroup_uncharge_end
| truncate_inode_pages_range
| truncate_inode_pages
| ext4_evict_inode
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--4.84%-- jbd2_journal_forget
| jbd2_journal_revoke
| __ext4_forget
| ext4_free_blocks
| ext4_ext_remove_space
| ext4_ext_truncate
| ext4_truncate
| ext4_evict_inode
| evict
| iput
| d_delete
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
|--4.84%-- __close_fd
| sys_close
| system_call
| __GI___libc_close
| main
| __libc_start_main
|
--4.84%-- dentry_lock_for_move
__d_move
d_move
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.90% loader1 [kernel.kallsyms] [k] _raw_spin_lock_irqsave
|
--- _raw_spin_lock_irqsave
|
|--47.17%-- audit_buffer_free
| audit_log_end
| |
| |--65.61%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--49.99%-- __GI___unlink
| | | |
| | | |--68.29%-- main
| | | | __libc_start_main
| | | |
| | | --31.71%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--34.16%-- truncate
| | | main
| | | __libc_start_main
| | |
| | --15.85%-- __GI___libc_chown
| | main
| | __libc_start_main
| |
| --34.39%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--34.89%-- rename
| | main
| | __libc_start_main
| |
| |--34.87%-- __GI___fchown
| | main
| | __libc_start_main
| |
| --30.25%-- __GI___unlink
| create_load
| main
| __libc_start_main
|
|--36.59%-- audit_log_start
| |
| |--73.18%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--39.46%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--21.13%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--21.08%-- rename
| | | main
| | | __libc_start_main
| | |
| | --18.32%-- __GI___chmod
| | main
| | __libc_start_main
| |
| --26.82%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| create_load
| main
| __libc_start_main
|
|--5.67%-- auditsc_get_stamp
| audit_log_start
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| truncate
| main
| __libc_start_main
|
|--5.66%-- audit_log_end
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __creat_nocancel
| main
| __libc_start_main
|
--4.91%-- __wake_up
jbd2_journal_stop
__ext4_journal_stop
ext4_da_write_end
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.88% loader1 [kernel.kallsyms] [k] do_get_write_access
|
--- do_get_write_access
jbd2_journal_get_write_access
__ext4_journal_get_write_access
|
|--63.98%-- ext4_reserve_inode_write
| |
| |--84.30%-- ext4_mark_inode_dirty
| | |
| | |--38.68%-- ext4_dirty_inode
| | | __mark_inode_dirty
| | | |
| | | |--75.92%-- ext4_setattr
| | | | notify_change
| | | | |
| | | | |--68.29%-- chmod_common
| | | | | sys_chmod
| | | | | system_call
| | | | | __GI___chmod
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --31.71%-- chown_common
| | | | sys_chown
| | | | system_call
| | | | __GI___libc_chown
| | | | main
| | | | __libc_start_main
| | | |
| | | --24.08%-- ext4_mb_new_blocks
| | | ext4_ext_map_blocks
| | | ext4_map_blocks
| | | ext4_getblk
| | | ext4_bread
| | | ext4_append
| | | ext4_mkdir
| | | vfs_mkdir
| | | sys_mkdir
| | | system_call
| | | __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--21.42%-- ext4_link
| | | vfs_link
| | | sys_link
| | | system_call
| | | __GI___link
| | | main
| | | __libc_start_main
| | |
| | |--21.28%-- ext4_rename
| | | vfs_rename
| | | SYSC_renameat
| | | sys_rename
| | | system_call
| | | rename
| | | main
| | | __libc_start_main
| | |
| | |--9.31%-- ext4_ext_tree_init
| | | __ext4_new_inode
| | | ext4_symlink
| | | vfs_symlink
| | | sys_symlink
| | | system_call
| | | __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | --9.31%-- add_dirent_to_buf
| | ext4_add_entry
| | ext4_add_nondir
| | ext4_symlink
| | vfs_symlink
| | sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| --15.70%-- ext4_xattr_set_handle
| ext4_xattr_set
| ext4_xattr_security_set
| generic_removexattr
| ima_inode_post_setattr
| notify_change
| |
| |--50.00%-- chmod_common
| | sys_fchmod
| | system_call
| | __GI___fchmod
| | main
| | __libc_start_main
| |
| --50.00%-- chown_common
| sys_fchown
| system_call
| __GI___fchown
| main
| __libc_start_main
|
|--20.19%-- __ext4_new_inode
| |
| |--74.73%-- ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| --25.27%-- ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
|--10.81%-- ext4_free_inode
| ext4_evict_inode
| evict
| iput
| |
| |--53.56%-- d_delete
| | vfs_rmdir
| | do_rmdir
| | sys_rmdir
| | system_call
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| --46.44%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--5.02%-- add_dirent_to_buf
ext4_add_entry
ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.81% loader1 [kernel.kallsyms] [k] __ext4_get_inode_loc
|
--- __ext4_get_inode_loc
|
|--93.71%-- ext4_get_inode_loc
| ext4_reserve_inode_write
| |
| |--64.20%-- ext4_mark_inode_dirty
| | |
| | |--19.52%-- ext4_rename
| | | vfs_rename
| | | SYSC_renameat
| | | sys_rename
| | | system_call
| | | rename
| | | main
| | | __libc_start_main
| | |
| | |--10.45%-- ext4_unlink
| | | vfs_unlink
| | | do_unlinkat
| | | sys_unlink
| | | system_call
| | | __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | |--10.45%-- ext4_evict_inode
| | | evict
| | | iput
| | | do_unlinkat
| | | sys_unlink
| | | system_call
| | | __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | |--10.45%-- ext4_ext_truncate
| | | ext4_truncate
| | | ext4_evict_inode
| | | evict
| | | iput
| | | d_delete
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--10.45%-- ext4_setattr
| | | notify_change
| | | do_truncate
| | | vfs_truncate
| | | do_sys_truncate
| | | sys_truncate
| | | system_call
| | | truncate
| | | main
| | | __libc_start_main
| | |
| | |--10.44%-- __ext4_ext_dirty
| | | ext4_ext_remove_space
| | | ext4_ext_truncate
| | | ext4_truncate
| | | ext4_evict_inode
| | | evict
| | | iput
| | | d_delete
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--10.11%-- ext4_dirty_inode
| | | __mark_inode_dirty
| | | ext4_setattr
| | | notify_change
| | | chmod_common
| | | sys_chmod
| | | system_call
| | | __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | |--9.06%-- add_dirent_to_buf
| | | ext4_add_entry
| | | ext4_add_nondir
| | | ext4_create
| | | vfs_create
| | | do_last
| | | path_openat
| | | do_filp_open
| | | do_sys_open
| | | sys_creat
| | | system_call
| | | __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | --9.06%-- __ext4_new_inode
| | ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| |--18.35%-- ext4_xattr_set_handle
| | ext4_xattr_set
| | ext4_xattr_security_set
| | generic_removexattr
| | ima_inode_post_setattr
| | notify_change
| | |
| | |--36.59%-- chown_common
| | | sys_chown
| | | system_call
| | | __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--31.71%-- do_truncate
| | | do_sys_ftruncate.constprop.13
| | | sys_ftruncate
| | | system_call
| | | __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | --31.71%-- chmod_common
| | sys_fchmod
| | system_call
| | __GI___fchmod
| | main
| | __libc_start_main
| |
| |--11.63%-- ext4_orphan_add
| | ext4_setattr
| | notify_change
| | do_truncate
| | vfs_truncate
| | do_sys_truncate
| | sys_truncate
| | system_call
| | truncate
| | main
| | __libc_start_main
| |
| --5.82%-- ext4_orphan_del
| ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--6.29%-- ext4_reserve_inode_write
ext4_mark_inode_dirty
add_dirent_to_buf
ext4_add_entry
ext4_add_nondir
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.80% loader1 [kernel.kallsyms] [k] audit_log_start
|
--- audit_log_start
|
|--54.43%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--23.25%-- __GI___ftruncate64
| | main
| | __libc_start_main
| |
| |--11.63%-- __GI___lchown
| | main
| | __libc_start_main
| |
| |--11.63%-- __GI___libc_close
| | main
| | __libc_start_main
| |
| |--11.63%-- __GI___unlink
| | main
| | __libc_start_main
| |
| |--11.62%-- __GI___fchmod
| | main
| | __libc_start_main
| |
| |--10.08%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--10.08%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| --10.08%-- rename
| main
| __libc_start_main
|
|--39.24%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--27.96%-- rename
| | main
| | __libc_start_main
| |
| |--27.96%-- __GI___mkdir
| | main
| | __libc_start_main
| |
| |--16.13%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--13.98%-- __GI___link
| | main
| | __libc_start_main
| |
| --13.98%-- __GI___symlink
| main
| __libc_start_main
|
--6.33%-- __audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.78% loader1 [kernel.kallsyms] [k] jbd2_journal_put_journal_head
|
--- jbd2_journal_put_journal_head
|
|--52.95%-- jbd2_journal_get_write_access
| __ext4_journal_get_write_access
| |
| |--55.68%-- ext4_reserve_inode_write
| | |
| | |--80.86%-- ext4_mark_inode_dirty
| | | |
| | | |--27.29%-- ext4_mkdir
| | | | vfs_mkdir
| | | | sys_mkdir
| | | | system_call
| | | | __GI___mkdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--25.36%-- __ext4_ext_dirty
| | | | ext4_ext_remove_space
| | | | ext4_ext_truncate
| | | | ext4_truncate
| | | | ext4_evict_inode
| | | | evict
| | | | iput
| | | | d_delete
| | | | vfs_rmdir
| | | | do_rmdir
| | | | sys_rmdir
| | | | system_call
| | | | __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--23.68%-- ext4_setattr
| | | | notify_change
| | | | do_truncate
| | | | vfs_truncate
| | | | do_sys_truncate
| | | | sys_truncate
| | | | system_call
| | | | truncate
| | | | main
| | | | __libc_start_main
| | | |
| | | --23.67%-- ext4_dirty_inode
| | | __mark_inode_dirty
| | | ext4_setattr
| | | notify_change
| | | chmod_common
| | | sys_fchmod
| | | system_call
| | | __GI___fchmod
| | | main
| | | __libc_start_main
| | |
| | --19.14%-- ext4_orphan_add
| | ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| |--23.01%-- __ext4_new_inode
| | |
| | |--53.45%-- ext4_mkdir
| | | vfs_mkdir
| | | sys_mkdir
| | | system_call
| | | __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | --46.55%-- ext4_create
| | vfs_create
| | do_last
| | path_openat
| | do_filp_open
| | do_sys_open
| | sys_creat
| | system_call
| | __creat_nocancel
| | main
| | __libc_start_main
| |
| |--10.66%-- ext4_orphan_add
| | ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| --10.66%-- ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--47.05%-- jbd2_journal_dirty_metadata
__ext4_handle_dirty_metadata
|
|--86.16%-- ext4_mark_iloc_dirty
| |
| |--55.98%-- ext4_mark_inode_dirty
| | |
| | |--25.39%-- ext4_dirty_inode
| | | __mark_inode_dirty
| | | ext4_mb_new_blocks
| | | ext4_ext_map_blocks
| | | ext4_map_blocks
| | | ext4_getblk
| | | ext4_bread
| | | ext4_append
| | | ext4_mkdir
| | | vfs_mkdir
| | | sys_mkdir
| | | system_call
| | | __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--24.88%-- ext4_ext_truncate
| | | ext4_truncate
| | | ext4_evict_inode
| | | evict
| | | iput
| | | d_delete
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--24.87%-- ext4_evict_inode
| | | evict
| | | iput
| | | do_unlinkat
| | | sys_unlink
| | | system_call
| | | __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | --24.87%-- ext4_setattr
| | notify_change
| | do_truncate
| | do_sys_ftruncate.constprop.13
| | sys_ftruncate
| | system_call
| | __GI___ftruncate64
| | main
| | __libc_start_main
| |
| |--29.98%-- ext4_orphan_del
| | ext4_evict_inode
| | evict
| | iput
| | d_delete
| | vfs_rmdir
| | do_rmdir
| | sys_rmdir
| | system_call
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| --14.04%-- ext4_orphan_add
| ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--13.84%-- ext4_handle_dirty_dirent_node
ext4_delete_entry
ext4_unlink
vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.78% loader1 [kernel.kallsyms] [k] put_dec
|
--- put_dec
|
|--88.66%-- number.isra.1
| vsnprintf
| audit_log_vformat
| audit_log_format
| |
| |--86.23%-- audit_log_start
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--39.54%-- __GI___unlink
| | | |
| | | |--78.38%-- main
| | | | __libc_start_main
| | | |
| | | --21.62%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--14.84%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--14.83%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | |--8.55%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--7.41%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--7.41%-- truncate
| | | main
| | | __libc_start_main
| | |
| | --7.41%-- __GI___mkdir
| | main
| | __libc_start_main
| |
| |--7.37%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | rename
| | main
| | __libc_start_main
| |
| --6.39%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| create_load
| main
| __libc_start_main
|
--11.34%-- vsnprintf
audit_log_vformat
audit_log_format
audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--50.00%-- __GI___fchmod
| main
| __libc_start_main
|
--50.00%-- __GI___libc_open
main
__libc_start_main
0.76% loader1 [kernel.kallsyms] [k] __alloc_skb
|
--- __alloc_skb
audit_log_start
|
|--81.72%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--22.51%-- __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| |--16.37%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--15.28%-- __GI___libc_write
| | main
| | __libc_start_main
| |
| |--14.19%-- __lxstat64
| | main
| | __libc_start_main
| |
| |--8.19%-- __GI___mkdir
| | main
| | __libc_start_main
| |
| |--8.19%-- __xstat64
| | main
| | __libc_start_main
| |
| |--8.18%-- __GI___lchown
| | main
| | __libc_start_main
| |
| --7.10%-- __GI___fchown
| main
| __libc_start_main
|
--18.28%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--68.28%-- __GI___unlink
| |
| |--53.53%-- create_load
| | main
| | __libc_start_main
| |
| --46.47%-- main
| __libc_start_main
|
--31.72%-- rename
main
__libc_start_main
0.75% loader1 [kernel.kallsyms] [k] kfree
|
--- kfree
|
|--56.69%-- skb_free_head
| skb_release_data
| skb_release_all
| kfree_skb
| audit_hold_skb
| audit_printk_skb
| audit_log_end
| |
| |--77.60%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--15.47%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--15.46%-- __GI___libc_close
| | | main
| | | __libc_start_main
| | |
| | |--15.46%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--13.40%-- __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--13.40%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | |--13.40%-- __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | --13.40%-- __GI___lchown
| | main
| | __libc_start_main
| |
| --22.40%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.56%-- __GI___unlink
| | main
| | __libc_start_main
| |
| --46.44%-- rename
| main
| __libc_start_main
|
|--37.41%-- audit_log_d_path
| |
| |--84.24%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--21.58%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--21.55%-- __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | |--19.45%-- __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--18.71%-- __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | --18.71%-- __GI___fchmod
| | main
| | __libc_start_main
| |
| --15.76%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __creat_nocancel
| main
| __libc_start_main
|
--5.90%-- ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.71% loader1 [kernel.kallsyms] [k] __ext4_check_dir_entry
|
--- __ext4_check_dir_entry
|
|--74.41%-- ext4_find_dest_de
| add_dirent_to_buf
| ext4_add_entry
| |
| |--36.93%-- ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| |--27.40%-- ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| |--27.39%-- ext4_add_nondir
| | |
| | |--65.11%-- ext4_create
| | | vfs_create
| | | do_last
| | | path_openat
| | | do_filp_open
| | | do_sys_open
| | | sys_creat
| | | system_call
| | | __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | --34.89%-- ext4_symlink
| | vfs_symlink
| | sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| --8.28%-- ext4_link
| vfs_link
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
|--13.27%-- ext4_generic_delete_entry
| ext4_delete_entry
| |
| |--53.57%-- ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| --46.43%-- ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--6.16%-- add_dirent_to_buf
| ext4_add_entry
| ext4_add_nondir
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--6.16%-- ext4_delete_entry
ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.71% loader1 [kernel.kallsyms] [k] start_this_handle
|
--- start_this_handle
jbd2__journal_start
__ext4_journal_start_sb
|
|--27.48%-- ext4_dirty_inode
| __mark_inode_dirty
| ext4_setattr
| notify_change
| |
| |--51.72%-- chown_common
| | |
| | |--50.01%-- sys_lchown
| | | system_call
| | | __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | --49.99%-- sys_chown
| | system_call
| | __GI___libc_chown
| | main
| | __libc_start_main
| |
| |--25.86%-- do_truncate
| | vfs_truncate
| | do_sys_truncate
| | sys_truncate
| | system_call
| | truncate
| | main
| | __libc_start_main
| |
| --22.42%-- chmod_common
| sys_fchmod
| system_call
| __GI___fchmod
| main
| __libc_start_main
|
|--20.38%-- ext4_xattr_set
| ext4_xattr_security_set
| generic_removexattr
| ima_inode_post_setattr
| notify_change
| |
| |--34.89%-- chown_common
| | sys_chown
| | system_call
| | __GI___libc_chown
| | main
| | __libc_start_main
| |
| |--34.88%-- do_truncate
| | do_sys_ftruncate.constprop.13
| | sys_ftruncate
| | system_call
| | __GI___ftruncate64
| | main
| | __libc_start_main
| |
| --30.24%-- chmod_common
| sys_fchmod
| system_call
| __GI___fchmod
| main
| __libc_start_main
|
|--14.21%-- ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--12.33%-- ext4_da_write_begin
| generic_file_buffered_write
| __generic_file_aio_write
| generic_file_aio_write
| ext4_file_write
| do_sync_write
| vfs_write
| sys_write
| system_call
| __GI___libc_write
| main
| __libc_start_main
|
|--7.11%-- ext4_link
| vfs_link
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
|--6.16%-- ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--6.16%-- ext4_evict_inode
| evict
| iput
| d_delete
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
--6.16%-- ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.70% loader1 [kernel.kallsyms] [k] __find_get_block
|
--- __find_get_block
|
|--93.72%-- __getblk
| |
| |--64.43%-- __ext4_get_inode_loc
| | ext4_get_inode_loc
| | ext4_reserve_inode_write
| | |
| | |--88.00%-- ext4_mark_inode_dirty
| | | |
| | | |--37.28%-- ext4_dirty_inode
| | | | __mark_inode_dirty
| | | | |
| | | | |--63.42%-- ext4_setattr
| | | | | notify_change
| | | | | |
| | | | | |--50.00%-- do_truncate
| | | | | | do_sys_ftruncate.constprop.13
| | | | | | sys_ftruncate
| | | | | | system_call
| | | | | | __GI___ftruncate64
| | | | | | main
| | | | | | __libc_start_main
| | | | | |
| | | | | --50.00%-- chown_common
| | | | | sys_lchown
| | | | | system_call
| | | | | __GI___lchown
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --36.58%-- ext4_free_blocks
| | | | ext4_ext_remove_space
| | | | ext4_ext_truncate
| | | | ext4_truncate
| | | | ext4_evict_inode
| | | | evict
| | | | iput
| | | | d_delete
| | | | vfs_rmdir
| | | | do_rmdir
| | | | sys_rmdir
| | | | system_call
| | | | __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--13.64%-- add_dirent_to_buf
| | | | ext4_add_entry
| | | | ext4_rename
| | | | vfs_rename
| | | | SYSC_renameat
| | | | sys_rename
| | | | system_call
| | | | rename
| | | | main
| | | | __libc_start_main
| | | |
| | | |--13.63%-- ext4_add_nondir
| | | | ext4_create
| | | | vfs_create
| | | | do_last
| | | | path_openat
| | | | do_filp_open
| | | | do_sys_open
| | | | sys_creat
| | | | system_call
| | | | __creat_nocancel
| | | | main
| | | | __libc_start_main
| | | |
| | | |--11.82%-- ext4_mkdir
| | | | vfs_mkdir
| | | | sys_mkdir
| | | | system_call
| | | | __GI___mkdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--11.82%-- ext4_evict_inode
| | | | evict
| | | | iput
| | | | d_delete
| | | | vfs_rmdir
| | | | do_rmdir
| | | | sys_rmdir
| | | | system_call
| | | | __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | --11.82%-- ext4_rename
| | | vfs_rename
| | | SYSC_renameat
| | | sys_rename
| | | system_call
| | | rename
| | | main
| | | __libc_start_main
| | |
| | --12.00%-- ext4_xattr_set_handle
| | ext4_xattr_set
| | ext4_xattr_security_set
| | generic_removexattr
| | ima_inode_post_setattr
| | notify_change
| | do_truncate
| | vfs_truncate
| | do_sys_truncate
| | sys_truncate
| | system_call
| | truncate
| | main
| | __libc_start_main
| |
| |--20.11%-- ext4_getblk
| | ext4_find_entry
| | |
| | |--66.67%-- ext4_unlink
| | | vfs_unlink
| | | do_unlinkat
| | | sys_unlink
| | | system_call
| | | __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | --33.33%-- ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| |--7.73%-- ext4_read_block_bitmap_nowait
| | ext4_read_block_bitmap
| | ext4_mb_mark_diskspace_used
| | ext4_mb_new_blocks
| | ext4_ext_map_blocks
| | ext4_map_blocks
| | ext4_getblk
| | ext4_bread
| | ext4_append
| | ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| --7.73%-- ext4_read_inode_bitmap
| __ext4_new_inode
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--6.28%-- ext4_free_blocks
ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.67% loader1 [kernel.kallsyms] [k] jbd2_journal_grab_journal_head
|
--- jbd2_journal_grab_journal_head
|
|--93.43%-- jbd2_journal_dirty_metadata
| |
| |--92.97%-- __ext4_handle_dirty_metadata
| | |
| | |--67.44%-- ext4_mark_iloc_dirty
| | | |
| | | |--75.87%-- ext4_mark_inode_dirty
| | | | |
| | | | |--17.06%-- ext4_ext_tree_init
| | | | | __ext4_new_inode
| | | | | ext4_symlink
| | | | | vfs_symlink
| | | | | sys_symlink
| | | | | system_call
| | | | | __GI___symlink
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | |--17.05%-- ext4_rename
| | | | | vfs_rename
| | | | | SYSC_renameat
| | | | | sys_rename
| | | | | system_call
| | | | | rename
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | |--17.04%-- ext4_ext_truncate
| | | | | ext4_truncate
| | | | | ext4_evict_inode
| | | | | evict
| | | | | iput
| | | | | d_delete
| | | | | vfs_rmdir
| | | | | do_rmdir
| | | | | sys_rmdir
| | | | | system_call
| | | | | __GI___rmdir
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | |--17.04%-- ext4_evict_inode
| | | | | evict
| | | | | iput
| | | | | do_unlinkat
| | | | | sys_unlink
| | | | | system_call
| | | | | __GI___unlink
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | |--17.03%-- ext4_mkdir
| | | | | vfs_mkdir
| | | | | sys_mkdir
| | | | | system_call
| | | | | __GI___mkdir
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --14.77%-- ext4_unlink
| | | | vfs_unlink
| | | | do_unlinkat
| | | | sys_unlink
| | | | system_call
| | | | __GI___unlink
| | | | main
| | | | __libc_start_main
| | | |
| | | |--12.93%-- ext4_orphan_del
| | | | ext4_truncate
| | | | ext4_setattr
| | | | notify_change
| | | | do_truncate
| | | | vfs_truncate
| | | | do_sys_truncate
| | | | sys_truncate
| | | | system_call
| | | | truncate
| | | | main
| | | | __libc_start_main
| | | |
| | | --11.21%-- ext4_orphan_add
| | | ext4_rmdir
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--16.28%-- __ext4_new_inode
| | | ext4_symlink
| | | vfs_symlink
| | | sys_symlink
| | | system_call
| | | __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--8.73%-- ext4_handle_dirty_dirent_node
| | | ext4_delete_entry
| | | ext4_rmdir
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | --7.56%-- ext4_mb_mark_diskspace_used
| | ext4_mb_new_blocks
| | ext4_ext_map_blocks
| | ext4_map_blocks
| | ext4_getblk
| | ext4_bread
| | ext4_append
| | ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| --7.03%-- __ext4_handle_dirty_super
| ext4_orphan_del
| ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--6.57%-- __ext4_handle_dirty_metadata
ext4_handle_dirty_dirent_node
ext4_delete_entry
ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.66% loader1 [kernel.kallsyms] [k] kmem_cache_alloc
|
--- kmem_cache_alloc
|
|--41.72%-- jbd2__journal_start
| __ext4_journal_start_sb
| |
| |--31.87%-- ext4_dirty_inode
| | __mark_inode_dirty
| | ext4_setattr
| | notify_change
| | |
| | |--50.00%-- chown_common
| | | sys_lchown
| | | system_call
| | | __GI___lchown
| | | main
| | | __libc_start_main
| | |
| | --50.00%-- chmod_common
| | sys_fchmod
| | system_call
| | __GI___fchmod
| | main
| | __libc_start_main
| |
| |--18.38%-- ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| |--17.88%-- ext4_xattr_set
| | ext4_xattr_security_set
| | generic_removexattr
| | ima_inode_post_setattr
| | notify_change
| | chmod_common
| | sys_fchmod
| | system_call
| | __GI___fchmod
| | main
| | __libc_start_main
| |
| |--15.94%-- __ext4_new_inode
| | ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| --15.94%-- ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--20.97%-- getname_flags
| |
| |--68.29%-- user_path_parent
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | |
| | |--53.57%-- main
| | | __libc_start_main
| | |
| | --46.43%-- create_load
| | main
| | __libc_start_main
| |
| --31.71%-- user_path_at_empty
| user_path_at
| sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
|--15.34%-- insert_revoke_hash
| jbd2_journal_revoke
| __ext4_forget
| ext4_free_blocks
| ext4_ext_remove_space
| ext4_ext_truncate
| ext4_truncate
| ext4_evict_inode
| evict
| iput
| d_delete
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
|--7.67%-- ext4_alloc_inode
| alloc_inode
| new_inode_pseudo
| new_inode
| __ext4_new_inode
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
|--7.65%-- __d_alloc
| d_alloc
| lookup_dcache
| __lookup_hash
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--6.65%-- __es_insert_extent
ext4_es_insert_extent
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.66% loader1 [kernel.kallsyms] [k] ___ratelimit
|
--- ___ratelimit
|
|--86.69%-- __printk_ratelimit
| |
| |--40.60%-- audit_log_lost
| | audit_printk_skb
| | audit_log_end
| | |
| | |--59.30%-- audit_log_name
| | | audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--68.13%-- rename
| | | | main
| | | | __libc_start_main
| | | |
| | | --31.87%-- __xstat64
| | | main
| | | __libc_start_main
| | |
| | --40.70%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--53.56%-- __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | --46.44%-- __GI___link
| | main
| | __libc_start_main
| |
| |--32.97%-- audit_panic
| | audit_log_lost
| | audit_printk_skb
| | audit_log_end
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--49.88%-- truncate
| | | main
| | | __libc_start_main
| | |
| | |--26.85%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | --23.28%-- __xstat64
| | main
| | __libc_start_main
| |
| --26.43%-- audit_printk_skb
| audit_log_end
| |
| |--66.51%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--50.36%-- __GI___fchown
| | | main
| | | __libc_start_main
| | |
| | --49.64%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| --33.49%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| create_load
| main
| __libc_start_main
|
|--6.65%-- audit_panic
| audit_log_lost
| audit_printk_skb
| audit_log_end
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___link
| main
| __libc_start_main
|
--6.65%-- audit_printk_skb
audit_log_end
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___rmdir
main
__libc_start_main
0.64% loader1 [kernel.kallsyms] [k] strlen
|
--- strlen
|
|--42.91%-- audit_log_untrustedstring
| |
| |--65.52%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--48.88%-- __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | |--26.68%-- __lxstat64
| | | main
| | | __libc_start_main
| | |
| | --24.45%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| |--18.46%-- audit_log_d_path
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| --16.02%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___fchown
| main
| __libc_start_main
|
|--36.48%-- audit_compare_dname_path
| __audit_inode_child
| |
| |--43.48%-- vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| |--18.84%-- vfs_link
| | sys_link
| | system_call
| | __GI___link
| | main
| | __libc_start_main
| |
| |--18.84%-- vfs_create
| | do_last
| | path_openat
| | do_filp_open
| | do_sys_open
| | sys_creat
| | system_call
| | __creat_nocancel
| | main
| | __libc_start_main
| |
| --18.84%-- may_delete
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--6.87%-- __audit_inode_child
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--6.87%-- audit_log_d_path
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| create_load
| main
| __libc_start_main
|
--6.87%-- parent_len
__audit_inode
filename_lookup
user_path_parent
do_unlinkat
sys_unlink
system_call
__GI___unlink
create_load
main
__libc_start_main
0.64% loader1 [kernel.kallsyms] [k] jbd2_journal_add_journal_head
|
--- jbd2_journal_add_journal_head
jbd2_journal_get_write_access
__ext4_journal_get_write_access
|
|--57.24%-- ext4_reserve_inode_write
| |
| |--63.79%-- ext4_mark_inode_dirty
| | |
| | |--21.76%-- ext4_mkdir
| | | vfs_mkdir
| | | sys_mkdir
| | | system_call
| | | __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | |--21.65%-- ext4_ext_truncate
| | | ext4_truncate
| | | ext4_evict_inode
| | | evict
| | | iput
| | | d_delete
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--18.86%-- ext4_dirty_inode
| | | __mark_inode_dirty
| | | generic_write_end
| | | ext4_da_write_end
| | | generic_file_buffered_write
| | | __generic_file_aio_write
| | | generic_file_aio_write
| | | ext4_file_write
| | | do_sync_write
| | | vfs_write
| | | sys_write
| | | system_call
| | | __GI___libc_write
| | | main
| | | __libc_start_main
| | |
| | |--18.86%-- add_dirent_to_buf
| | | ext4_add_entry
| | | ext4_rename
| | | vfs_rename
| | | SYSC_renameat
| | | sys_rename
| | | system_call
| | | rename
| | | main
| | | __libc_start_main
| | |
| | --18.86%-- ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --36.21%-- ext4_orphan_add
| |
| |--33.54%-- ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| |--33.23%-- ext4_rmdir
| | vfs_rmdir
| | do_rmdir
| | sys_rmdir
| | system_call
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| --33.23%-- ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
|--22.10%-- add_dirent_to_buf
| ext4_add_entry
| |
| |--64.02%-- ext4_add_nondir
| | |
| | |--51.32%-- ext4_symlink
| | | vfs_symlink
| | | sys_symlink
| | | system_call
| | | __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | --48.68%-- ext4_create
| | vfs_create
| | do_last
| | path_openat
| | do_filp_open
| | do_sys_open
| | sys_creat
| | system_call
| | __creat_nocancel
| | main
| | __libc_start_main
| |
| --35.98%-- ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--6.89%-- ext4_orphan_add
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
|--6.89%-- __ext4_new_inode
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--6.88%-- ext4_free_inode
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.60% loader1 [kernel.kallsyms] [k] put_dec_full9
|
--- put_dec_full9
number.isra.1
vsnprintf
audit_log_vformat
audit_log_format
audit_log_start
|
|--53.95%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--28.71%-- __GI___symlink
| | main
| | __libc_start_main
| |
| |--15.61%-- __GI___fchmod
| | main
| | __libc_start_main
| |
| |--14.63%-- __GI___link
| | main
| | __libc_start_main
| |
| |--13.89%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| |--13.63%-- __GI___unlink
| | main
| | __libc_start_main
| |
| --13.53%-- __creat_nocancel
| main
| __libc_start_main
|
--46.05%-- audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--34.15%-- __lxstat64
| main
| __libc_start_main
|
|--18.29%-- __GI___fchown
| main
| __libc_start_main
|
|--15.85%-- __GI___unlink
| main
| __libc_start_main
|
|--15.85%-- __creat_nocancel
| main
| __libc_start_main
|
--15.85%-- __GI___chmod
main
__libc_start_main
0.55% loader1 [kernel.kallsyms] [k] kmem_cache_free
|
--- kmem_cache_free
|
|--44.08%-- kfree_skbmem
| kfree_skb
| audit_hold_skb
| audit_printk_skb
| audit_log_end
| |
| |--60.57%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--34.89%-- __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | |--34.88%-- __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | --30.23%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --39.43%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.56%-- __GI___chmod
| | main
| | __libc_start_main
| |
| --46.44%-- __GI___link
| main
| __libc_start_main
|
|--29.22%-- final_putname
| __audit_syscall_exit
| sysret_audit
| |
| |--39.73%-- __GI___symlink
| | main
| | __libc_start_main
| |
| |--31.88%-- truncate
| | main
| | __libc_start_main
| |
| --28.39%-- __GI___libc_chown
| main
| __libc_start_main
|
|--9.32%-- jbd2_journal_stop
| __ext4_journal_stop
| ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--9.31%-- ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--8.07%-- __ext4_journal_stop
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.54% loader1 [kernel.kallsyms] [k] _raw_spin_trylock
|
--- _raw_spin_trylock
|
|--49.33%-- ___ratelimit
| __printk_ratelimit
| |
| |--49.97%-- audit_panic
| | audit_log_lost
| | audit_printk_skb
| | audit_log_end
| | |
| | |--66.67%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--50.00%-- __GI___link
| | | | main
| | | | __libc_start_main
| | | |
| | | --50.00%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | --33.33%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___lchown
| | main
| | __libc_start_main
| |
| |--33.37%-- audit_printk_skb
| | audit_log_end
| | audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| --16.66%-- audit_log_lost
| audit_printk_skb
| audit_log_end
| audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __xstat64
| main
| __libc_start_main
|
|--32.98%-- __printk_ratelimit
| |
| |--75.08%-- audit_printk_skb
| | audit_log_end
| | |
| | |--66.37%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | __GI___unlink
| | | |
| | | |--50.00%-- main
| | | | __libc_start_main
| | | |
| | | --50.00%-- create_load
| | | main
| | | __libc_start_main
| | |
| | --33.63%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___link
| | main
| | __libc_start_main
| |
| --24.92%-- audit_log_lost
| audit_printk_skb
| audit_log_end
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___fchown
| main
| __libc_start_main
|
|--9.48%-- ext4_mb_regular_allocator
| ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--8.22%-- ext4_read_inode_bitmap
__ext4_new_inode
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.51% loader1 [kernel.kallsyms] [k] memcmp
|
--- memcmp
|
|--72.88%-- search_dir
| ext4_find_entry
| |
| |--49.37%-- ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| |--27.03%-- ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| --23.60%-- ext4_lookup
| lookup_real
| __lookup_hash
| |
| |--50.01%-- SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --49.99%-- kern_path_create
| user_path_create
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
|--18.52%-- ext4_find_dest_de
| add_dirent_to_buf
| ext4_add_entry
| |
| |--53.57%-- ext4_link
| | vfs_link
| | sys_link
| | system_call
| | __GI___link
| | main
| | __libc_start_main
| |
| --46.43%-- ext4_add_nondir
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--8.60%-- ext4_find_entry
ext4_unlink
vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.50% loader1 [kernel.kallsyms] [k] jbd2_journal_dirty_metadata
|
--- jbd2_journal_dirty_metadata
|
|--81.20%-- __ext4_handle_dirty_metadata
| |
| |--78.51%-- ext4_mark_iloc_dirty
| | |
| | |--84.22%-- ext4_mark_inode_dirty
| | | |
| | | |--32.51%-- ext4_dirty_inode
| | | | __mark_inode_dirty
| | | | |
| | | | |--50.00%-- ext4_mb_new_blocks
| | | | | ext4_ext_map_blocks
| | | | | ext4_map_blocks
| | | | | ext4_getblk
| | | | | ext4_bread
| | | | | ext4_append
| | | | | ext4_mkdir
| | | | | vfs_mkdir
| | | | | sys_mkdir
| | | | | system_call
| | | | | __GI___mkdir
| | | | | main
| | | | | __libc_start_main
| | | | |
| | | | --50.00%-- ext4_setattr
| | | | notify_change
| | | | chown_common
| | | | sys_lchown
| | | | system_call
| | | | __GI___lchown
| | | | main
| | | | __libc_start_main
| | | |
| | | |--18.74%-- ext4_add_nondir
| | | | ext4_symlink
| | | | vfs_symlink
| | | | sys_symlink
| | | | system_call
| | | | __GI___symlink
| | | | main
| | | | __libc_start_main
| | | |
| | | |--16.25%-- ext4_rmdir
| | | | vfs_rmdir
| | | | do_rmdir
| | | | sys_rmdir
| | | | system_call
| | | | __GI___rmdir
| | | | main
| | | | __libc_start_main
| | | |
| | | |--16.25%-- ext4_evict_inode
| | | | evict
| | | | iput
| | | | do_unlinkat
| | | | sys_unlink
| | | | system_call
| | | | __GI___unlink
| | | | main
| | | | __libc_start_main
| | | |
| | | --16.25%-- ext4_unlink
| | | vfs_unlink
| | | do_unlinkat
| | | sys_unlink
| | | system_call
| | | __GI___unlink
| | | main
| | | __libc_start_main
| | |
| | --15.78%-- ext4_orphan_add
| | ext4_rmdir
| | vfs_rmdir
| | do_rmdir
| | sys_rmdir
| | system_call
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| |--10.75%-- ext4_free_inode
| | ext4_evict_inode
| | evict
| | iput
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| --10.75%-- __ext4_new_inode
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--18.80%-- __ext4_handle_dirty_super
ext4_orphan_del
|
|--53.57%-- ext4_evict_inode
| evict
| iput
| d_delete
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
--46.43%-- ext4_truncate
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.49% loader1 [kernel.kallsyms] [k] memset
|
--- memset
|
|--39.02%-- ext4_block_truncate_page
| ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| |
| |--53.56%-- do_sys_ftruncate.constprop.13
| | sys_ftruncate
| | system_call
| | __GI___ftruncate64
| | main
| | __libc_start_main
| |
| --46.44%-- vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
|--29.97%-- audit_log_start
| |
| |--69.76%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--50.04%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | --49.96%-- __GI___link
| | main
| | __libc_start_main
| |
| --30.24%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___fchown
| main
| __libc_start_main
|
|--20.56%-- truncate_pagecache
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--10.45%-- jbd2__journal_start
__ext4_journal_start_sb
ext4_dirty_inode
__mark_inode_dirty
ext4_setattr
notify_change
chmod_common
sys_fchmod
system_call
__GI___fchmod
main
__libc_start_main
0.48% loader1 [kernel.kallsyms] [k] audit_log_format
|
--- audit_log_format
|
|--40.53%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--52.34%-- __lxstat64
| | main
| | __libc_start_main
| |
| |--24.96%-- __GI___libc_close
| | main
| | __libc_start_main
| |
| --22.69%-- __creat_nocancel
| main
| __libc_start_main
|
|--19.85%-- audit_log_start
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.49%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| --46.51%-- __GI___libc_open
| main
| __libc_start_main
|
|--19.80%-- __audit_syscall_exit
| sysret_audit
| |
| |--53.56%-- __GI___libc_write
| | main
| | __libc_start_main
| |
| --46.44%-- __GI___symlink
| main
| __libc_start_main
|
|--10.62%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___rmdir
| main
| __libc_start_main
|
--9.20%-- audit_log_key
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
main
__libc_start_main
0.47% loader1 [kernel.kallsyms] [k] xattr_resolve_name
|
--- xattr_resolve_name
|
|--70.72%-- generic_getxattr
| get_vfs_caps_from_disk
| audit_copy_inode
| |
| |--84.84%-- __audit_inode
| | |
| | |--69.05%-- filename_lookup
| | | user_path_parent
| | | |
| | | |--51.73%-- do_unlinkat
| | | | sys_unlink
| | | | system_call
| | | | __GI___unlink
| | | | create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --48.27%-- SYSC_renameat
| | | sys_rename
| | | system_call
| | | rename
| | | main
| | | __libc_start_main
| | |
| | |--15.48%-- do_last
| | | path_openat
| | | do_filp_open
| | | do_sys_open
| | | sys_creat
| | | system_call
| | | __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | --15.48%-- sys_fchown
| | system_call
| | __GI___fchown
| | main
| | __libc_start_main
| |
| --15.16%-- __audit_inode_child
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--29.28%-- generic_removexattr
ima_inode_post_setattr
notify_change
|
|--36.57%-- chown_common
| sys_fchown
| system_call
| __GI___fchown
| main
| __libc_start_main
|
|--31.72%-- do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--31.71%-- chmod_common
sys_fchmod
system_call
__GI___fchmod
main
__libc_start_main
0.47% loader1 [kernel.kallsyms] [k] audit_log_exit
|
--- audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--29.73%-- rename
| main
| __libc_start_main
|
|--20.25%-- __GI___unlink
| main
| __libc_start_main
|
|--18.85%-- __GI___mkdir
| main
| __libc_start_main
|
|--10.88%-- __GI___libc_write
| main
| __libc_start_main
|
|--10.87%-- truncate
| main
| __libc_start_main
|
--9.43%-- __GI___fchown
main
__libc_start_main
0.45% loader1 [kernel.kallsyms] [k] prepend_path
|
--- prepend_path
|
|--90.15%-- d_path
| audit_log_d_path
| |
| |--56.30%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--38.82%-- __GI___unlink
| | | |
| | | |--50.02%-- main
| | | | __libc_start_main
| | | |
| | | --49.98%-- create_load
| | | main
| | | __libc_start_main
| | |
| | |--22.38%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | |--19.40%-- __GI___link
| | | main
| | | __libc_start_main
| | |
| | --19.40%-- __GI___ftruncate64
| | main
| | __libc_start_main
| |
| --43.70%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--25.00%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--25.00%-- __GI___mkdir
| | main
| | __libc_start_main
| |
| |--25.00%-- rename
| | main
| | __libc_start_main
| |
| --25.00%-- __GI___unlink
| create_load
| main
| __libc_start_main
|
--9.85%-- audit_log_d_path
audit_log_exit
__audit_syscall_exit
sysret_audit
__creat_nocancel
main
__libc_start_main
0.44% loader1 [kernel.kallsyms] [k] jbd2_journal_cancel_revoke
|
--- jbd2_journal_cancel_revoke
do_get_write_access
jbd2_journal_get_write_access
__ext4_journal_get_write_access
|
|--90.05%-- ext4_reserve_inode_write
| |
| |--34.82%-- ext4_orphan_add
| | |
| | |--63.43%-- ext4_setattr
| | | notify_change
| | | do_truncate
| | | vfs_truncate
| | | do_sys_truncate
| | | sys_truncate
| | | system_call
| | | truncate
| | | main
| | | __libc_start_main
| | |
| | --36.57%-- ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| |--32.04%-- ext4_mark_inode_dirty
| | |
| | |--60.24%-- ext4_ext_truncate
| | | ext4_truncate
| | | ext4_setattr
| | | notify_change
| | | do_truncate
| | | do_sys_ftruncate.constprop.13
| | | sys_ftruncate
| | | system_call
| | | __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | --39.76%-- add_dirent_to_buf
| | ext4_add_entry
| | ext4_add_nondir
| | ext4_symlink
| | vfs_symlink
| | sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| |--22.09%-- ext4_xattr_set_handle
| | ext4_xattr_set
| | ext4_xattr_security_set
| | generic_removexattr
| | ima_inode_post_setattr
| | notify_change
| | |
| | |--50.00%-- chown_common
| | | sys_chown
| | | system_call
| | | __GI___libc_chown
| | | main
| | | __libc_start_main
| | |
| | --50.00%-- do_truncate
| | do_sys_ftruncate.constprop.13
| | sys_ftruncate
| | system_call
| | __GI___ftruncate64
| | main
| | __libc_start_main
| |
| --11.04%-- ext4_orphan_del
| ext4_evict_inode
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--9.95%-- ext4_orphan_add
ext4_setattr
notify_change
do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.44% loader1 [kernel.kallsyms] [k] audit_log_n_untrustedstring
|
--- audit_log_n_untrustedstring
|
|--68.10%-- audit_log_untrustedstring
| |
| |--53.17%-- audit_log_d_path
| | |
| | |--56.44%-- audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--50.50%-- __GI___libc_open
| | | | main
| | | | __libc_start_main
| | | |
| | | --49.50%-- __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | --43.56%-- audit_log_task_info
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--64.13%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | --35.87%-- rename
| | main
| | __libc_start_main
| |
| --46.83%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--36.57%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| |--31.72%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| --31.72%-- __creat_nocancel
| main
| __libc_start_main
|
|--21.79%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| main
| __libc_start_main
|
--10.11%-- audit_log_d_path
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.43% loader1 [kernel.kallsyms] [k] search_dir
|
--- search_dir
ext4_find_entry
|
|--42.53%-- ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--33.86%-- ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--23.62%-- ext4_lookup
lookup_real
__lookup_hash
|
|--50.01%-- SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--49.99%-- kern_path_create
user_path_create
sys_link
system_call
__GI___link
main
__libc_start_main
0.43% loader1 [kernel.kallsyms] [k] prepend_name.isra.9
|
--- prepend_name.isra.9
prepend_path
d_path
audit_log_d_path
|
|--66.15%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--33.33%-- __GI___libc_close
| | main
| | __libc_start_main
| |
| |--30.96%-- __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| |--17.86%-- __GI___link
| | main
| | __libc_start_main
| |
| --17.85%-- __GI___libc_chown
| main
| __libc_start_main
|
--33.85%-- audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--65.12%-- __GI___unlink
| create_load
| main
| __libc_start_main
|
--34.88%-- __GI___chmod
main
__libc_start_main
0.39% loader1 [kernel.kallsyms] [k] kmem_cache_alloc_node
|
--- kmem_cache_alloc_node
|
|--87.04%-- __alloc_skb
| audit_log_start
| |
| |--74.19%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--20.08%-- __GI___libc_open
| | | main
| | | __libc_start_main
| | |
| | |--20.07%-- rename
| | | main
| | | __libc_start_main
| | |
| | |--20.06%-- __GI___symlink
| | | main
| | | __libc_start_main
| | |
| | |--20.05%-- __GI___ftruncate64
| | | main
| | | __libc_start_main
| | |
| | --19.74%-- __GI___lchown
| | main
| | __libc_start_main
| |
| --25.81%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--50.00%-- __creat_nocancel
| | main
| | __libc_start_main
| |
| --50.00%-- __GI___fchmod
| main
| __libc_start_main
|
--12.96%-- audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.38% swapper [kernel.kallsyms] [k] pm_qos_request
|
--- pm_qos_request
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.38% loader1 [kernel.kallsyms] [k] ksize
|
--- ksize
|
|--63.37%-- __alloc_skb
| audit_log_start
| |
| |--60.63%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--65.14%-- __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | --34.86%-- __GI___fchmod
| | main
| | __libc_start_main
| |
| --39.37%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.44%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --46.56%-- __GI___lchown
| main
| __libc_start_main
|
|--23.23%-- audit_log_start
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--50.00%-- __GI___libc_write
| | main
| | __libc_start_main
| |
| --50.00%-- __creat_nocancel
| main
| __libc_start_main
|
--13.40%-- kzfree
apparmor_file_free_security
security_file_free
__fput
____fput
task_work_run
do_notify_resume
int_signal
__GI___libc_close
main
__libc_start_main
0.37% loader1 [kernel.kallsyms] [k] bit_waitqueue
|
--- bit_waitqueue
|
|--49.31%-- unlock_buffer
| |
| |--75.93%-- do_get_write_access
| | jbd2_journal_get_write_access
| | __ext4_journal_get_write_access
| | |
| | |--68.29%-- ext4_reserve_inode_write
| | | ext4_mark_inode_dirty
| | | |
| | | |--53.57%-- ext4_dirty_inode
| | | | __mark_inode_dirty
| | | | ext4_setattr
| | | | notify_change
| | | | do_truncate
| | | | do_sys_ftruncate.constprop.13
| | | | sys_ftruncate
| | | | system_call
| | | | __GI___ftruncate64
| | | | main
| | | | __libc_start_main
| | | |
| | | --46.43%-- ext4_rmdir
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | --31.71%-- add_dirent_to_buf
| | ext4_add_entry
| | ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| --24.07%-- ll_rw_block
| ext4_find_entry
| ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--25.61%-- __inode_wait_for_writeback
| inode_wait_for_writeback
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--25.08%-- wake_up_bit
unlock_buffer
do_get_write_access
jbd2_journal_get_write_access
__ext4_journal_get_write_access
ext4_reserve_inode_write
ext4_mark_inode_dirty
|
|--52.66%-- ext4_dirty_inode
| __mark_inode_dirty
| ext4_setattr
| notify_change
| chown_common
| sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
--47.34%-- ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.37% loader1 [kernel.kallsyms] [k] skip_atoi
|
--- skip_atoi
format_decode
vsnprintf
audit_log_vformat
audit_log_format
|
|--62.25%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--22.16%-- __GI___link
| | main
| | __libc_start_main
| |
| |--20.12%-- __xstat64
| | main
| | __libc_start_main
| |
| |--19.29%-- __GI___lchown
| | main
| | __libc_start_main
| |
| |--19.22%-- rename
| | main
| | __libc_start_main
| |
| --19.22%-- __GI___symlink
| main
| __libc_start_main
|
--37.75%-- audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--68.30%-- __lxstat64
| main
| __libc_start_main
|
--31.70%-- __GI___unlink
create_load
main
__libc_start_main
0.36% swapper [kernel.kallsyms] [k] enqueue_entity
|
--- enqueue_entity
enqueue_task_fair
enqueue_task
activate_task
ttwu_do_activate.constprop.74
try_to_wake_up
wake_up_process
hrtimer_wakeup
__run_hrtimer
hrtimer_interrupt
local_apic_timer_interrupt
smp_apic_timer_interrupt
apic_timer_interrupt
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.36% kworker/u64:0 [kernel.kallsyms] [k] find_next_bit
|
--- find_next_bit
cpumask_next_and
find_busiest_group
load_balance
idle_balance
__schedule
schedule
worker_thread
kthread
ret_from_fork
0.34% loader1 [kernel.kallsyms] [k] audit_log_task_info
|
--- audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--27.90%-- __GI___unlink
| main
| __libc_start_main
|
|--14.83%-- __GI___fchown
| main
| __libc_start_main
|
|--14.81%-- __lxstat64
| main
| __libc_start_main
|
|--14.81%-- __GI___link
| main
| __libc_start_main
|
|--14.81%-- __GI___libc_write
| main
| __libc_start_main
|
--12.84%-- __GI___chmod
main
__libc_start_main
0.34% monitor [kernel.kallsyms] [k] native_read_cr0
|
--- native_read_cr0
__libc_waitpid
0.34% loader1 [kernel.kallsyms] [k] jbd2_journal_stop
|
--- jbd2_journal_stop
__ext4_journal_stop
|
|--28.13%-- ext4_dirty_inode
| __mark_inode_dirty
| ext4_setattr
| notify_change
| chown_common
| sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
|--28.13%-- ext4_create
| vfs_create
| do_last
| path_openat
| do_filp_open
| do_sys_open
| sys_creat
| system_call
| __creat_nocancel
| main
| __libc_start_main
|
|--15.08%-- ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
|--15.07%-- ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--13.60%-- ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.34% loader1 [kernel.kallsyms] [k] audit_buffer_free
|
--- audit_buffer_free
|
|--84.88%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--33.49%-- __creat_nocancel
| | main
| | __libc_start_main
| |
| |--30.89%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| |--17.81%-- rename
| | main
| | __libc_start_main
| |
| --17.81%-- __GI___ftruncate64
| main
| __libc_start_main
|
--15.12%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
main
__libc_start_main
0.34% loader1 [kernel.kallsyms] [k] __ext4_handle_dirty_metadata
|
--- __ext4_handle_dirty_metadata
|
|--41.40%-- ext4_mark_iloc_dirty
| ext4_mark_inode_dirty
| |
| |--36.56%-- ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| |--31.72%-- ext4_ext_truncate
| | ext4_truncate
| | ext4_setattr
| | notify_change
| | do_truncate
| | do_sys_ftruncate.constprop.13
| | sys_ftruncate
| | system_call
| | __GI___ftruncate64
| | main
| | __libc_start_main
| |
| --31.72%-- ext4_rmdir
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
|--30.28%-- __ext4_new_inode
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
|--15.15%-- ext4_orphan_add
| ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--13.17%-- ext4_handle_dirty_dirent_node
ext4_delete_entry
ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.33% loader1 [kernel.kallsyms] [k] __brelse
|
--- __brelse
|
|--59.99%-- ext4_mark_inode_dirty
| |
| |--25.87%-- add_dirent_to_buf
| | ext4_add_entry
| | ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| |--25.86%-- ext4_dirty_inode
| | __mark_inode_dirty
| | ext4_setattr
| | notify_change
| | chmod_common
| | sys_chmod
| | system_call
| | __GI___chmod
| | main
| | __libc_start_main
| |
| |--25.85%-- ext4_unlink
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| --22.42%-- ext4_add_nondir
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
|--15.51%-- __getblk
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
|--13.45%-- ext4_orphan_add
| ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--11.05%-- ext4_xattr_set
ext4_xattr_security_set
generic_removexattr
ima_inode_post_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.32% loader1 [kernel.kallsyms] [k] d_path
|
--- d_path
audit_log_d_path
|
|--86.32%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--68.29%-- __GI___unlink
| | |
| | |--53.57%-- create_load
| | | main
| | | __libc_start_main
| | |
| | --46.43%-- main
| | __libc_start_main
| |
| |--15.85%-- __GI___libc_close
| | main
| | __libc_start_main
| |
| --15.85%-- __GI___libc_chown
| main
| __libc_start_main
|
--13.68%-- audit_log_exit
__audit_syscall_exit
sysret_audit
rename
main
__libc_start_main
0.32% swapper [kernel.kallsyms] [k] native_read_tsc
|
--- native_read_tsc
ktime_get
cpuidle_enter_state
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.32% loader1 [kernel.kallsyms] [k] string.isra.5
|
--- string.isra.5
vsnprintf
audit_log_vformat
audit_log_format
|
|--55.91%-- audit_log_d_path
| |
| |--50.00%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--50.00%-- __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | --50.00%-- __GI___libc_open
| | main
| | __libc_start_main
| |
| --50.00%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--50.00%-- __GI___libc_chown
| | main
| | __libc_start_main
| |
| --50.00%-- __GI___symlink
| main
| __libc_start_main
|
|--30.11%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.58%-- __creat_nocancel
| | main
| | __libc_start_main
| |
| --46.42%-- __GI___libc_close
| main
| __libc_start_main
|
--13.98%-- audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___libc_close
main
__libc_start_main
0.32% loader1 [kernel.kallsyms] [k] unlock_buffer
|
--- unlock_buffer
do_get_write_access
jbd2_journal_get_write_access
__ext4_journal_get_write_access
|
|--69.90%-- ext4_reserve_inode_write
| |
| |--80.00%-- ext4_mark_inode_dirty
| | |
| | |--25.00%-- __ext4_ext_dirty
| | | ext4_ext_remove_space
| | | ext4_ext_truncate
| | | ext4_truncate
| | | ext4_evict_inode
| | | evict
| | | iput
| | | d_delete
| | | vfs_rmdir
| | | do_rmdir
| | | sys_rmdir
| | | system_call
| | | __GI___rmdir
| | | main
| | | __libc_start_main
| | |
| | |--25.00%-- ext4_setattr
| | | notify_change
| | | do_truncate
| | | vfs_truncate
| | | do_sys_truncate
| | | sys_truncate
| | | system_call
| | | truncate
| | | main
| | | __libc_start_main
| | |
| | |--25.00%-- ext4_ext_tree_init
| | | __ext4_new_inode
| | | ext4_create
| | | vfs_create
| | | do_last
| | | path_openat
| | | do_filp_open
| | | do_sys_open
| | | sys_creat
| | | system_call
| | | __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | --25.00%-- ext4_dirty_inode
| | __mark_inode_dirty
| | ext4_mb_new_blocks
| | ext4_ext_map_blocks
| | ext4_map_blocks
| | ext4_getblk
| | ext4_bread
| | ext4_append
| | ext4_mkdir
| | vfs_mkdir
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| --20.00%-- ext4_orphan_add
| ext4_rmdir
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
|--16.12%-- ext4_delete_entry
| ext4_rmdir
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
--13.98%-- ext4_free_inode
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.30% swapper [kernel.kallsyms] [k] rcu_eqs_exit_common.isra.49
|
--- rcu_eqs_exit_common.isra.49
rcu_eqs_exit
rcu_idle_exit
cpu_startup_entry
start_secondary
0.29% loader1 [kernel.kallsyms] [k] __mnt_want_write
|
--- __mnt_want_write
mnt_want_write
|
|--69.57%-- kern_path_create
| user_path_create
| |
| |--75.01%-- sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| --24.99%-- sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
|--15.36%-- sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
--15.08%-- SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.29% loader1 [kernel.kallsyms] [k] __kmalloc_node_track_caller
|
--- __kmalloc_node_track_caller
|
|--82.57%-- __kmalloc_reserve.isra.26
| __alloc_skb
| audit_log_start
| |
| |--57.78%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--68.16%-- __GI___chmod
| | | main
| | | __libc_start_main
| | |
| | --31.84%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --42.22%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| rename
| main
| __libc_start_main
|
--17.43%-- __alloc_skb
audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.29% loader1 [kernel.kallsyms] [k] link_path_walk
|
--- link_path_walk
|
|--82.56%-- path_lookupat
| filename_lookup
| |
| |--60.57%-- user_path_parent
| | |
| | |--69.77%-- do_unlinkat
| | | sys_unlink
| | | system_call
| | | __GI___unlink
| | | create_load
| | | main
| | | __libc_start_main
| | |
| | --30.23%-- SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --39.43%-- user_path_at_empty
| user_path_at
| |
| |--53.56%-- sys_lchown
| | system_call
| | __GI___lchown
| | main
| | __libc_start_main
| |
| --46.44%-- do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--17.44%-- path_openat
do_filp_open
do_sys_open
sys_open
system_call
__GI___libc_open
main
__libc_start_main
0.28% loader1 [kernel.kallsyms] [k] audit_log_n_string
|
--- audit_log_n_string
|
|--82.15%-- audit_log_n_untrustedstring
| |
| |--81.16%-- audit_log_untrustedstring
| | audit_log_d_path
| | |
| | |--76.78%-- audit_log_task_info
| | | audit_log_exit
| | | __audit_syscall_exit
| | | sysret_audit
| | | |
| | | |--34.88%-- __GI___unlink
| | | | main
| | | | __libc_start_main
| | | |
| | | |--34.88%-- __GI___libc_close
| | | | create_load
| | | | main
| | | | __libc_start_main
| | | |
| | | --30.24%-- __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | --23.22%-- audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __lxstat64
| | main
| | __libc_start_main
| |
| --18.84%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___mkdir
| main
| __libc_start_main
|
--17.85%-- audit_log_untrustedstring
audit_log_d_path
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.28% loader1 [kernel.kallsyms] [k] __sb_start_write
|
--- __sb_start_write
|
|--68.85%-- mnt_want_write
| |
| |--26.04%-- sys_lchown
| | system_call
| | __GI___lchown
| | main
| | __libc_start_main
| |
| |--25.83%-- kern_path_create
| | user_path_create
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| |--25.56%-- chmod_common
| | sys_chmod
| | system_call
| | __GI___chmod
| | main
| | __libc_start_main
| |
| --22.57%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| create_load
| main
| __libc_start_main
|
|--15.61%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--15.54%-- vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.28% loader1 [kernel.kallsyms] [k] __ext4_journal_get_write_access
|
--- __ext4_journal_get_write_access
|
|--34.14%-- ext4_reserve_inode_write
| ext4_mark_inode_dirty
| |
| |--53.56%-- __ext4_ext_dirty
| | ext4_ext_remove_space
| | ext4_ext_truncate
| | ext4_truncate
| | ext4_setattr
| | notify_change
| | do_truncate
| | vfs_truncate
| | do_sys_truncate
| | sys_truncate
| | system_call
| | truncate
| | main
| | __libc_start_main
| |
| --46.44%-- ext4_ext_truncate
| ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
|--18.31%-- ext4_evict_inode
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--15.85%-- ext4_free_inode
| ext4_evict_inode
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--15.85%-- ext4_mark_inode_dirty
| add_dirent_to_buf
| ext4_add_entry
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--15.85%-- ext4_orphan_add
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.27% loader1 [kernel.kallsyms] [k] ext4_find_dest_de
|
--- ext4_find_dest_de
add_dirent_to_buf
ext4_add_entry
|
|--34.97%-- ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--32.56%-- ext4_link
| vfs_link
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
--32.47%-- ext4_add_nondir
ext4_symlink
vfs_symlink
sys_symlink
system_call
__GI___symlink
main
__libc_start_main
0.27% loader1 [kernel.kallsyms] [k] crc16
|
--- crc16
|
|--83.75%-- ext4_group_desc_csum
| ext4_group_desc_csum_set
| |
| |--80.59%-- __ext4_new_inode
| | |
| | |--75.92%-- ext4_mkdir
| | | vfs_mkdir
| | | sys_mkdir
| | | system_call
| | | __GI___mkdir
| | | main
| | | __libc_start_main
| | |
| | --24.08%-- ext4_symlink
| | vfs_symlink
| | sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| --19.41%-- ext4_mb_mark_diskspace_used
| ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--16.25%-- ext4_group_desc_csum_set
ext4_free_inode
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.25% loader1 [kernel.kallsyms] [k] audit_filter_inodes
|
--- audit_filter_inodes
__audit_syscall_exit
sysret_audit
|
|--20.55%-- __GI___libc_write
| main
| __libc_start_main
|
|--20.55%-- __GI___fchmod
| main
| __libc_start_main
|
|--20.55%-- __GI___ftruncate64
| main
| __libc_start_main
|
|--20.54%-- __GI___lchown
| main
| __libc_start_main
|
--17.81%-- rename
main
__libc_start_main
0.24% loader1 [kernel.kallsyms] [k] ext4_ext_remove_space
|
--- ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
|
|--57.74%-- ext4_evict_inode
| evict
| iput
| d_delete
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
--42.26%-- ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.24% loader1 [kernel.kallsyms] [k] do_last
|
--- do_last
path_openat
do_filp_open
do_sys_open
|
|--78.87%-- sys_creat
| system_call
| __creat_nocancel
| main
| __libc_start_main
|
--21.13%-- sys_open
system_call
__GI___libc_open
main
__libc_start_main
0.24% loader1 [kernel.kallsyms] [k] ext4_getblk
|
--- ext4_getblk
|
|--57.87%-- ext4_find_entry
| ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--42.13%-- ext4_bread
__ext4_read_dirblock
ext4_add_entry
|
|--50.23%-- ext4_link
| vfs_link
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
--49.77%-- ext4_add_nondir
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.23% loader1 [kernel.kallsyms] [k] system_call
|
--- system_call
|
|--21.74%-- __GI___fchmod
| main
| __libc_start_main
|
|--21.73%-- __GI___libc_write
| main
| __libc_start_main
|
|--18.84%-- truncate
| main
| __libc_start_main
|
|--18.84%-- __creat_nocancel
| main
| __libc_start_main
|
--18.84%-- __GI___libc_close
create_load
main
__libc_start_main
0.23% loader1 [kernel.kallsyms] [k] __audit_inode_child
|
--- __audit_inode_child
|
|--38.80%-- may_delete
| |
| |--50.00%-- vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --50.00%-- vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--22.40%-- vfs_link
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
|--19.40%-- vfs_create
| do_last
| path_openat
| do_filp_open
| do_sys_open
| sys_creat
| system_call
| __creat_nocancel
| main
| __libc_start_main
|
--19.40%-- vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.23% loader1 [kernel.kallsyms] [k] up_read
|
--- up_read
|
|--58.20%-- ext4_xattr_get
| ext4_xattr_security_get
| generic_getxattr
| get_vfs_caps_from_disk
| audit_copy_inode
| |
| |--66.67%-- __audit_inode_child
| | may_delete
| | |
| | |--50.00%-- vfs_rename
| | | SYSC_renameat
| | | sys_rename
| | | system_call
| | | rename
| | | main
| | | __libc_start_main
| | |
| | --50.00%-- vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| --33.33%-- __audit_inode
| filename_lookup
| user_path_at_empty
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
|--22.40%-- ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--19.40%-- audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___fchmod
main
__libc_start_main
0.23% loader1 [kernel.kallsyms] [k] skb_put
|
--- skb_put
|
|--80.60%-- audit_log_format
| |
| |--51.86%-- audit_log_start
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | __GI___libc_close
| | main
| | __libc_start_main
| |
| --48.14%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___lchown
| main
| __libc_start_main
|
--19.40%-- audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___fchmod
main
__libc_start_main
0.22% loader1 [kernel.kallsyms] [k] __audit_inode
|
--- __audit_inode
|
|--60.00%-- filename_lookup
| |
| |--66.67%-- kern_path_create
| | user_path_create
| | sys_mkdir
| | system_call
| | __GI___mkdir
| | main
| | __libc_start_main
| |
| --33.33%-- user_path_at_empty
| user_path_at
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
|--20.00%-- sys_fchmod
| system_call
| __GI___fchmod
| main
| __libc_start_main
|
--20.00%-- system_call
__GI___fchmod
main
__libc_start_main
0.22% loader1 [kernel.kallsyms] [k] audit_log_name
|
--- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--40.00%-- __GI___unlink
| |
| |--50.00%-- create_load
| | main
| | __libc_start_main
| |
| --50.00%-- main
| __libc_start_main
|
|--20.00%-- truncate
| main
| __libc_start_main
|
|--20.00%-- __GI___rmdir
| main
| __libc_start_main
|
--20.00%-- __GI___lchown
main
__libc_start_main
0.20% loader1 [kernel.kallsyms] [k] ext4_journal_check_start
|
--- ext4_journal_check_start
__ext4_journal_start_sb
|
|--25.87%-- ext4_xattr_set
| ext4_xattr_security_set
| generic_removexattr
| ima_inode_post_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
|--25.86%-- ext4_rmdir
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
|--25.85%-- ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--22.42%-- __ext4_new_inode
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.20% loader1 [kernel.kallsyms] [k] ext4_reserve_inode_write
|
--- ext4_reserve_inode_write
|
|--74.19%-- ext4_mark_inode_dirty
| |
| |--34.89%-- ext4_evict_inode
| | evict
| | iput
| | d_delete
| | vfs_rmdir
| | do_rmdir
| | sys_rmdir
| | system_call
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| |--34.88%-- add_dirent_to_buf
| | ext4_add_entry
| | ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --30.24%-- ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--25.81%-- ext4_orphan_add
ext4_setattr
notify_change
do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.19% loader1 [kernel.kallsyms] [k] __nlmsg_put
|
--- __nlmsg_put
|
|--73.20%-- audit_log_start
| |
| |--63.42%-- audit_log_name
| | audit_log_exit
| | __audit_syscall_exit
| | sysret_audit
| | |
| | |--50.00%-- rename
| | | main
| | | __libc_start_main
| | |
| | --50.00%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --36.58%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___symlink
| main
| __libc_start_main
|
--26.80%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.19% loader1 [kernel.kallsyms] [k] ext4_mb_complex_scan_group
|
--- ext4_mb_complex_scan_group
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.19% loader1 [kernel.kallsyms] [k] ext4_generic_delete_entry
|
--- ext4_generic_delete_entry
ext4_delete_entry
|
|--53.57%-- ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--23.22%-- ext4_unlink
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--23.22%-- ext4_rmdir
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.19% loader1 [kernel.kallsyms] [k] __call_rcu
|
--- __call_rcu
call_rcu_sched
|
|--76.62%-- ext4_destroy_inode
| destroy_inode
| evict
| iput
| |
| |--64.79%-- d_delete
| | vfs_rmdir
| | do_rmdir
| | sys_rmdir
| | system_call
| | __GI___rmdir
| | main
| | __libc_start_main
| |
| --35.21%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--23.38%-- __fput
____fput
task_work_run
do_notify_resume
int_signal
__GI___libc_close
main
__libc_start_main
0.19% loader1 [kernel.kallsyms] [k] audit_printk_skb
|
--- audit_printk_skb
audit_log_end
|
|--52.61%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--51.94%-- __GI___libc_close
| | create_load
| | main
| | __libc_start_main
| |
| |--45.04%-- __GI___lchown
| | main
| | __libc_start_main
| |
| --3.02%-- __GI___unlink
| create_load
| main
| __libc_start_main
|
--47.39%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
rename
main
__libc_start_main
0.18% loader1 [kernel.kallsyms] [k] __wake_up_bit
|
--- __wake_up_bit
|
|--52.33%-- unlock_buffer
| do_get_write_access
| jbd2_journal_get_write_access
| __ext4_journal_get_write_access
| ext4_reserve_inode_write
| ext4_mark_inode_dirty
| |
| |--52.61%-- add_dirent_to_buf
| | ext4_add_entry
| | ext4_rename
| | vfs_rename
| | SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --47.39%-- __ext4_new_inode
| ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--47.67%-- wake_up_bit
unlock_buffer
do_get_write_access
jbd2_journal_get_write_access
__ext4_journal_get_write_access
ext4_reserve_inode_write
ext4_mark_inode_dirty
|
|--50.00%-- ext4_dirty_inode
| __mark_inode_dirty
| generic_write_end
| ext4_da_write_end
| generic_file_buffered_write
| __generic_file_aio_write
| generic_file_aio_write
| ext4_file_write
| do_sync_write
| vfs_write
| sys_write
| system_call
| __GI___libc_write
| main
| __libc_start_main
|
--50.00%-- ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.18% loader1 libc-2.19.so [.] __GI___libc_close
|
--- __GI___libc_close
|
|--72.25%-- main
| __libc_start_main
|
--27.75%-- create_load
main
__libc_start_main
0.18% loader1 [kernel.kallsyms] [k] ext4_alloc_inode
|
--- ext4_alloc_inode
|
|--51.86%-- new_inode_pseudo
| new_inode
| __ext4_new_inode
| ext4_create
| vfs_create
| do_last
| path_openat
| do_filp_open
| do_sys_open
| sys_creat
| system_call
| __creat_nocancel
| main
| __libc_start_main
|
--48.14%-- alloc_inode
new_inode_pseudo
new_inode
__ext4_new_inode
|
|--50.00%-- ext4_symlink
| vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--50.00%-- ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.18% loader1 [kernel.kallsyms] [k] __ext4_journal_start_sb
|
--- __ext4_journal_start_sb
|
|--27.77%-- ext4_ext_remove_space
| ext4_ext_truncate
| ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
|--24.08%-- ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
|--24.08%-- ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--24.08%-- vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.18% loader1 [kernel.kallsyms] [k] add_dirent_to_buf
|
--- add_dirent_to_buf
|
|--75.92%-- ext4_add_entry
| |
| |--68.29%-- ext4_add_nondir
| | |
| | |--53.56%-- ext4_create
| | | vfs_create
| | | do_last
| | | path_openat
| | | do_filp_open
| | | do_sys_open
| | | sys_creat
| | | system_call
| | | __creat_nocancel
| | | main
| | | __libc_start_main
| | |
| | --46.44%-- ext4_symlink
| | vfs_symlink
| | sys_symlink
| | system_call
| | __GI___symlink
| | main
| | __libc_start_main
| |
| --31.71%-- ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--24.08%-- ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.18% loader1 [kernel.kallsyms] [k] __ext4_new_inode
|
--- __ext4_new_inode
ext4_symlink
vfs_symlink
sys_symlink
system_call
__GI___symlink
main
__libc_start_main
0.18% loader1 [kernel.kallsyms] [k] truncate_inode_pages_range
|
--- truncate_inode_pages_range
|
|--50.00%-- truncate_pagecache
| ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--50.00%-- truncate_inode_pages
ext4_evict_inode
evict
iput
|
|--50.00%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--50.00%-- d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.18% loader1 [kernel.kallsyms] [k] __audit_syscall_exit
|
--- __audit_syscall_exit
|
|--75.00%-- sysret_audit
| |
| |--66.67%-- __GI___unlink
| | |
| | |--50.00%-- main
| | | __libc_start_main
| | |
| | --50.00%-- create_load
| | main
| | __libc_start_main
| |
| --33.33%-- __GI___ftruncate64
| main
| __libc_start_main
|
--25.00%-- __GI___symlink
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] ext4_mark_inode_dirty
|
--- ext4_mark_inode_dirty
|
|--28.64%-- vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
|--28.64%-- ext4_evict_inode
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--28.64%-- ext4_ext_remove_space
| ext4_ext_truncate
| ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--14.09%-- ext4_dirty_inode
__mark_inode_dirty
ext4_setattr
notify_change
chmod_common
sys_fchmod
system_call
__GI___fchmod
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] strncmp
|
--- strncmp
audit_compare_dname_path
__audit_inode_child
|
|--66.67%-- vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--33.33%-- vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] audit_log_key
|
--- audit_log_key
|
|--66.66%-- __audit_syscall_exit
| sysret_audit
| |
| |--50.03%-- __GI___unlink
| | main
| | __libc_start_main
| |
| --49.97%-- __GI___ftruncate64
| main
| __libc_start_main
|
--33.34%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___chmod
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] notify_change
|
--- notify_change
chown_common
|
|--66.67%-- sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
--33.33%-- sys_lchown
system_call
__GI___lchown
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] find_next_zero_bit
|
--- find_next_zero_bit
ext4_mb_complex_scan_group
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] audit_log_end
|
--- audit_log_end
|
|--68.46%-- __audit_syscall_exit
| sysret_audit
| |
| |--50.01%-- __GI___unlink
| | main
| | __libc_start_main
| |
| --49.99%-- __GI___chmod
| main
| __libc_start_main
|
--31.54%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___libc_chown
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] ext4_free_inode
|
--- ext4_free_inode
ext4_evict_inode
evict
iput
|
|--69.77%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--30.23%-- d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] parent_len
|
--- parent_len
__audit_inode
filename_lookup
user_path_parent
do_unlinkat
sys_unlink
system_call
__GI___unlink
|
|--65.11%-- create_load
| main
| __libc_start_main
|
--34.89%-- main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] get_vfs_caps_from_disk
|
--- get_vfs_caps_from_disk
|
|--65.11%-- audit_copy_inode
| |
| |--53.56%-- __audit_inode_child
| | may_delete
| | vfs_unlink
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | main
| | __libc_start_main
| |
| --46.44%-- __audit_inode
| filename_lookup
| user_path_at_empty
| user_path_at
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--34.89%-- __audit_inode
filename_lookup
user_path_at_empty
user_path_at
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] path_init
|
--- path_init
|
|--69.76%-- path_lookupat
| filename_lookup
| user_path_parent
| |
| |--50.00%-- do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| --50.00%-- SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--30.24%-- filename_lookup
user_path_at_empty
user_path_at
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] kfree_skb
|
--- kfree_skb
audit_hold_skb
audit_printk_skb
audit_log_end
|
|--65.11%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.56%-- __GI___rmdir
| | main
| | __libc_start_main
| |
| --46.44%-- __GI___mkdir
| main
| __libc_start_main
|
--34.89%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] task_tgid_nr_ns
|
--- task_tgid_nr_ns
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--34.88%-- __GI___libc_chown
| main
| __libc_start_main
|
|--34.88%-- __GI___mkdir
| main
| __libc_start_main
|
--30.24%-- __GI___unlink
create_load
main
__libc_start_main
0.15% loader1 [kernel.kallsyms] [k] ext4_setattr
|
--- ext4_setattr
notify_change
|
|--34.88%-- chmod_common
| sys_chmod
| system_call
| __GI___chmod
| main
| __libc_start_main
|
|--34.88%-- chown_common
| sys_lchown
| system_call
| __GI___lchown
| main
| __libc_start_main
|
--30.25%-- do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.14% swapper [kernel.kallsyms] [k] tick_check_idle
|
--- tick_check_idle
irq_enter
do_IRQ
ret_from_intr
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.14% loader1 [kernel.kallsyms] [k] _raw_spin_lock_irq
|
--- _raw_spin_lock_irq
|
|--68.29%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| |
| |--53.57%-- __GI___symlink
| | main
| | __libc_start_main
| |
| --46.43%-- __GI___unlink
| main
| __libc_start_main
|
--31.71%-- __schedule
_cond_resched
down_read
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
rename
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] mb_find_order_for_block
|
--- mb_find_order_for_block
|
|--68.29%-- mb_find_extent
| ext4_mb_complex_scan_group
| ext4_mb_regular_allocator
| ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--31.71%-- ext4_mb_complex_scan_group
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] complete_walk
|
--- complete_walk
path_lookupat
filename_lookup
|
|--68.29%-- user_path_parent
| |
| |--53.57%-- SYSC_renameat
| | sys_rename
| | system_call
| | rename
| | main
| | __libc_start_main
| |
| --46.43%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| create_load
| main
| __libc_start_main
|
--31.71%-- user_path_at_empty
user_path_at
sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] audit_log_lost
|
--- audit_log_lost
audit_printk_skb
audit_log_end
|
|--68.29%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___link
| main
| __libc_start_main
|
--31.71%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
create_load
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] ext4_discard_preallocations
|
--- ext4_discard_preallocations
|
|--36.58%-- ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
|--31.71%-- ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--31.71%-- ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] audit_filter_syscall
|
--- audit_filter_syscall
|
|--36.58%-- __audit_syscall_exit
| sysret_audit
| __GI___libc_write
| main
| __libc_start_main
|
|--31.71%-- auditsys
| __GI___rmdir
| main
| __libc_start_main
|
--31.71%-- __audit_syscall_entry
auditsys
__xstat64
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] generic_permission
|
--- generic_permission
|
|--68.29%-- __inode_permission
| inode_permission
| |
| |--53.56%-- link_path_walk
| | path_lookupat
| | filename_lookup
| | user_path_parent
| | do_unlinkat
| | sys_unlink
| | system_call
| | __GI___unlink
| | create_load
| | main
| | __libc_start_main
| |
| --46.44%-- vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--31.71%-- inode_permission
link_path_walk
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] ima_inode_post_setattr
|
--- ima_inode_post_setattr
notify_change
|
|--68.29%-- chown_common
| sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
--31.71%-- do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] audit_alloc_name
|
--- audit_alloc_name
__audit_inode_child
|
|--36.57%-- vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
|--31.72%-- vfs_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--31.72%-- vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.14% loader1 [kernel.kallsyms] [k] from_kgid
|
--- from_kgid
|
|--68.29%-- ext4_mark_inode_dirty
| ext4_dirty_inode
| __mark_inode_dirty
| ext4_setattr
| notify_change
| chmod_common
| sys_chmod
| system_call
| __GI___chmod
| main
| __libc_start_main
|
--31.71%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
main
__libc_start_main
0.13% loader1 [kernel.kallsyms] [k] do_unlinkat
|
--- do_unlinkat
sys_unlink
system_call
__GI___unlink
|
|--66.67%-- main
| __libc_start_main
|
--33.33%-- create_load
main
__libc_start_main
0.13% loader1 [kernel.kallsyms] [k] audit_copy_inode
|
--- audit_copy_inode
|
|--66.67%-- __audit_inode_child
| may_delete
| vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--33.33%-- __audit_inode
filename_lookup
user_path_parent
do_unlinkat
sys_unlink
system_call
__GI___unlink
create_load
main
__libc_start_main
0.13% loader1 [kernel.kallsyms] [k] strncpy
|
--- strncpy
get_task_comm
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--33.33%-- __GI___symlink
| main
| __libc_start_main
|
|--33.33%-- __GI___unlink
| main
| __libc_start_main
|
--33.33%-- truncate
main
__libc_start_main
0.13% loader1 [kernel.kallsyms] [k] audit_filter_rules.isra.7
|
--- audit_filter_rules.isra.7
audit_filter_syscall
__audit_syscall_exit
sysret_audit
|
|--33.33%-- __GI___rmdir
| main
| __libc_start_main
|
|--33.33%-- __creat_nocancel
| main
| __libc_start_main
|
--33.33%-- __xstat64
main
__libc_start_main
0.13% loader1 [kernel.kallsyms] [k] kmem_cache_alloc_trace
|
--- kmem_cache_alloc_trace
audit_log_d_path
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--33.33%-- __lxstat64
| main
| __libc_start_main
|
|--33.33%-- __GI___symlink
| main
| __libc_start_main
|
--33.33%-- __GI___mkdir
main
__libc_start_main
0.13% loader1 [kernel.kallsyms] [k] common_perm
|
--- common_perm
|
|--33.35%-- apparmor_path_unlink
| security_path_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
|--33.35%-- common_perm_cond
| apparmor_path_chown
| security_path_chown
| chown_common
| sys_lchown
| system_call
| __GI___lchown
| main
| __libc_start_main
|
--33.30%-- common_perm_rm.isra.15.constprop.22
apparmor_path_unlink
security_path_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.13% rcuos/9 [kernel.kallsyms] [k] __free_pages
|
--- __free_pages
__free_memcg_kmem_pages
__free_slab
discard_slab
unfreeze_partials.isra.44
__slab_free
kmem_cache_free
file_free_rcu
rcu_nocb_kthread
kthread
ret_from_fork
0.13% rcuos/9 [kernel.kallsyms] [k] __slab_free
|
--- __slab_free
kmem_cache_free
ext4_i_callback
rcu_nocb_kthread
kthread
ret_from_fork
0.12% loader1 [kernel.kallsyms] [k] __kmalloc_reserve.isra.26
|
--- __kmalloc_reserve.isra.26
|
|--87.56%-- audit_log_start
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___rmdir
| main
| __libc_start_main
|
--12.44%-- __alloc_skb
audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___mkdir
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] vfs_rename
|
--- vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] ext4_group_desc_csum
|
--- ext4_group_desc_csum
|
|--50.00%-- ext4_group_desc_csum_set
| ext4_mb_mark_diskspace_used
| ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--50.00%-- __ext4_new_inode
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] current_kernel_time
|
--- current_kernel_time
|
|--50.01%-- add_dirent_to_buf
| ext4_add_entry
| ext4_add_nondir
| ext4_create
| vfs_create
| do_last
| path_openat
| do_filp_open
| do_sys_open
| sys_creat
| system_call
| __creat_nocancel
| main
| __libc_start_main
|
--49.99%-- ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] ext4_map_blocks
|
--- ext4_map_blocks
ext4_getblk
ext4_find_entry
|
|--50.02%-- ext4_lookup
| lookup_real
| __lookup_hash
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--49.98%-- ext4_unlink
vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] map_id_up
|
--- map_id_up
|
|--50.00%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __creat_nocancel
| main
| __libc_start_main
|
--50.00%-- from_kuid
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
create_load
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] __read_lock_failed
|
--- __read_lock_failed
_raw_read_lock
start_this_handle
jbd2__journal_start
__ext4_journal_start_sb
|
|--50.00%-- __ext4_new_inode
| ext4_create
| vfs_create
| do_last
| path_openat
| do_filp_open
| do_sys_open
| sys_creat
| system_call
| __creat_nocancel
| main
| __libc_start_main
|
--50.00%-- ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] __ext4_journal_stop
|
--- __ext4_journal_stop
ext4_dirty_inode
__mark_inode_dirty
ext4_setattr
notify_change
do_truncate
|
|--50.01%-- do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--49.99%-- vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] mutex_lock
|
--- mutex_lock
|
|--50.01%-- SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--49.99%-- sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] chmod_common
|
--- chmod_common
|
|--50.70%-- sys_chmod
| system_call
| __GI___chmod
| main
| __libc_start_main
|
--49.30%-- system_call
__GI___chmod
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] get_page_from_freelist
|
--- get_page_from_freelist
__alloc_pages_nodemask
alloc_pages_current
__page_cache_alloc
grab_cache_page_write_begin
ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.10% loader1 [kernel.kallsyms] [k] lockref_put_or_lock
|
--- lockref_put_or_lock
dput
|
|--53.25%-- SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--46.75%-- do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_create
|
--- ext4_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __remove_inode_hash
|
--- __remove_inode_hash
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_ext_map_blocks
|
--- ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] security_inode_permission
|
--- security_inode_permission
inode_permission
link_path_walk
|
|--53.57%-- path_lookupat
| filename_lookup
| user_path_parent
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| create_load
| main
| __libc_start_main
|
--46.43%-- path_openat
do_filp_open
do_sys_open
sys_open
system_call
__GI___libc_open
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] common_perm_create.constprop.23
|
--- common_perm_create.constprop.23
|
|--53.57%-- security_path_symlink
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--46.43%-- apparmor_path_mknod
security_path_mknod
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_es_free_extent
|
--- ext4_es_free_extent
ext4_es_remove_extent
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_get_group_desc
|
--- ext4_get_group_desc
|
|--53.57%-- ext4_get_inode_loc
| ext4_reserve_inode_write
| ext4_mark_inode_dirty
| ext4_dirty_inode
| __mark_inode_dirty
| ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--46.43%-- ext4_symlink
vfs_symlink
sys_symlink
system_call
__GI___symlink
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __inode_permission
|
--- __inode_permission
inode_permission
|
|--53.57%-- link_path_walk
| path_lookupat
| filename_lookup
| user_path_at_empty
| user_path_at
| vfs_fstatat
| SYSC_newstat
| sys_newstat
| system_call
| __xstat64
| main
| __libc_start_main
|
--46.43%-- vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_block_bitmap_csum_set
|
--- ext4_block_bitmap_csum_set
|
|--53.57%-- ext4_mb_mark_diskspace_used
| ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--46.43%-- ext4_free_blocks
ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] audit_putname
|
--- audit_putname
|
|--53.57%-- user_path_create
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
--46.43%-- do_unlinkat
sys_unlink
system_call
__GI___unlink
create_load
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __getblk
|
--- __getblk
|
|--53.57%-- __ext4_get_inode_loc
| ext4_get_inode_loc
| ext4_reserve_inode_write
| ext4_mark_inode_dirty
| add_dirent_to_buf
| ext4_add_entry
| ext4_rename
| vfs_rename
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--46.43%-- ext4_read_inode_bitmap
ext4_free_inode
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.09% loader1 libc-2.19.so [.] rename
|
--- rename
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] d_walk
|
--- d_walk
shrink_dcache_parent
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __d_lookup_rcu
|
--- __d_lookup_rcu
|
|--53.56%-- path_lookupat
| filename_lookup
| user_path_at_empty
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
--46.44%-- lookup_fast
path_lookupat
filename_lookup
user_path_at_empty
user_path_at
sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] auditsc_get_stamp
|
--- auditsc_get_stamp
audit_log_start
|
|--53.56%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___symlink
| main
| __libc_start_main
|
--46.44%-- audit_log_exit
__audit_syscall_exit
sysret_audit
truncate
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] unmap_mapping_range
|
--- unmap_mapping_range
ext4_setattr
notify_change
do_truncate
|
|--53.56%-- vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--46.44%-- do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] from_kuid
|
--- from_kuid
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--53.56%-- rename
| main
| __libc_start_main
|
--46.44%-- __GI___link
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] inode_init_owner
|
--- inode_init_owner
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_truncate
|
--- ext4_truncate
|
|--53.56%-- ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--46.44%-- ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] create_page_buffers
|
--- create_page_buffers
|
|--53.56%-- __block_write_begin
| ext4_da_write_begin
| generic_file_buffered_write
| __generic_file_aio_write
| generic_file_aio_write
| ext4_file_write
| do_sync_write
| vfs_write
| sys_write
| system_call
| __GI___libc_write
| main
| __libc_start_main
|
--46.44%-- ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_xattr_ibody_get
|
--- ext4_xattr_ibody_get
|
|--53.56%-- ext4_xattr_security_get
| generic_getxattr
| get_vfs_caps_from_disk
| audit_copy_inode
| __audit_inode
| filename_lookup
| user_path_parent
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| create_load
| main
| __libc_start_main
|
--46.44%-- ext4_xattr_get
ext4_xattr_security_get
generic_getxattr
get_vfs_caps_from_disk
audit_copy_inode
__audit_inode
filename_lookup
user_path_parent
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] security_inode_unlink
|
--- security_inode_unlink
|
|--53.56%-- do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--46.44%-- vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] block_invalidatepage
|
--- block_invalidatepage
ext4_invalidatepage
ext4_da_invalidatepage
truncate_inode_pages_range
truncate_pagecache
ext4_setattr
notify_change
do_truncate
|
|--53.56%-- do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--46.44%-- vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] ext4_has_inline_data
|
--- ext4_has_inline_data
|
|--53.56%-- ext4_rmdir
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
--46.44%-- ext4_unlink
vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] vfs_getattr
|
--- vfs_getattr
|
|--53.56%-- SYSC_newlstat
| sys_newlstat
| system_call
| __lxstat64
| main
| __libc_start_main
|
--46.44%-- vfs_fstatat
SYSC_newstat
sys_newstat
system_call
__xstat64
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] strncpy_from_user
|
--- strncpy_from_user
|
|--53.56%-- getname_flags
| user_path_create
| sys_symlink
| system_call
| __GI___symlink
| main
| __libc_start_main
|
--46.44%-- user_path_parent
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] skb_release_data
|
--- skb_release_data
skb_release_all
kfree_skb
audit_hold_skb
audit_printk_skb
audit_log_end
|
|--53.56%-- audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___link
| main
| __libc_start_main
|
--46.44%-- audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___link
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] next_zones_zonelist
|
--- next_zones_zonelist
|
|--53.56%-- __alloc_pages_nodemask
| alloc_pages_current
| __page_cache_alloc
| grab_cache_page_write_begin
| ext4_da_write_begin
| generic_file_buffered_write
| __generic_file_aio_write
| generic_file_aio_write
| ext4_file_write
| do_sync_write
| vfs_write
| sys_write
| system_call
| __GI___libc_write
| main
| __libc_start_main
|
--46.44%-- __slab_alloc
kmem_cache_alloc_node
__alloc_skb
audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___libc_open
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] strnlen
|
--- strnlen
string.isra.5
vsnprintf
audit_log_vformat
audit_log_format
|
|--53.55%-- audit_log_task_info
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| main
| __libc_start_main
|
--46.45%-- audit_log_d_path
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__creat_nocancel
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] sb_is_blkdev_sb
|
--- sb_is_blkdev_sb
|
|--53.53%-- ext4_mb_new_blocks
| ext4_ext_map_blocks
| ext4_map_blocks
| ext4_getblk
| ext4_bread
| ext4_append
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--46.47%-- evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] path_get
|
--- path_get
vfs_open
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __fsnotify_parent
|
--- __fsnotify_parent
|
|--50.16%-- vfs_unlink
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--49.84%-- notify_change
chown_common
sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __jbd2_journal_file_buffer
|
--- __jbd2_journal_file_buffer
jbd2_journal_dirty_metadata
__ext4_handle_dirty_metadata
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] path_lookupat
|
--- path_lookupat
|
|--50.02%-- user_path_parent
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--49.98%-- filename_lookup
user_path_at_empty
user_path_at
sys_chmod
system_call
__GI___chmod
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] mutex_unlock
|
--- mutex_unlock
|
|--50.01%-- ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--49.99%-- SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __percpu_counter_add
|
--- __percpu_counter_add
|
|--50.01%-- ext4_free_inode
| ext4_evict_inode
| evict
| iput
| do_unlinkat
| sys_unlink
| system_call
| __GI___unlink
| main
| __libc_start_main
|
--49.99%-- ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] audit_filter_type
|
--- audit_filter_type
audit_log_exit
__audit_syscall_exit
sysret_audit
|
|--50.01%-- __GI___libc_close
| main
| __libc_start_main
|
--49.99%-- __GI___unlink
create_load
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __wake_up
|
--- __wake_up
jbd2_journal_stop
__ext4_journal_stop
|
|--50.01%-- ext4_truncate
| ext4_setattr
| notify_change
| do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--49.99%-- ext4_da_write_end
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] kmalloc_slab
|
--- kmalloc_slab
__kmalloc_reserve.isra.26
__alloc_skb
audit_log_start
|
|--50.01%-- audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___unlink
| main
| __libc_start_main
|
--49.99%-- audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __mark_inode_dirty
|
--- __mark_inode_dirty
|
|--50.00%-- ext4_setattr
| notify_change
| chown_common
| sys_chown
| system_call
| __GI___libc_chown
| main
| __libc_start_main
|
--50.00%-- ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] zone_dirty_ok
|
--- zone_dirty_ok
get_page_from_freelist
__alloc_pages_nodemask
alloc_pages_current
__page_cache_alloc
grab_cache_page_write_begin
ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] radix_tree_next_chunk
|
--- radix_tree_next_chunk
|
|--50.00%-- pagevec_lookup
| truncate_inode_pages_range
| truncate_pagecache
| ext4_setattr
| notify_change
| do_truncate
| vfs_truncate
| do_sys_truncate
| sys_truncate
| system_call
| truncate
| main
| __libc_start_main
|
--50.00%-- find_get_pages
pagevec_lookup
truncate_inode_pages_range
truncate_pagecache
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] _raw_spin_unlock
|
--- _raw_spin_unlock
|
|--50.00%-- d_lookup
| lookup_dcache
| __lookup_hash
| SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--50.00%-- ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __block_commit_write.isra.21
|
--- __block_commit_write.isra.21
block_write_end
generic_write_end
ext4_da_write_end
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] getname_flags
|
--- getname_flags
user_path_at_empty
user_path_at
|
|--50.00%-- sys_lchown
| system_call
| __GI___lchown
| main
| __libc_start_main
|
--50.00%-- sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] mnt_drop_write
|
--- mnt_drop_write
|
|--50.00%-- system_call
| __GI___libc_chown
| main
| __libc_start_main
|
--50.00%-- do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __es_remove_extent
|
--- __es_remove_extent
ext4_es_remove_extent
|
|--50.00%-- ext4_ext_truncate
| ext4_truncate
| ext4_evict_inode
| evict
| iput
| d_delete
| vfs_rmdir
| do_rmdir
| sys_rmdir
| system_call
| __GI___rmdir
| main
| __libc_start_main
|
--50.00%-- ext4_clear_inode
ext4_free_inode
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] _raw_read_lock
|
--- _raw_read_lock
start_this_handle
jbd2__journal_start
__ext4_journal_start_sb
|
|--50.00%-- ext4_link
| vfs_link
| sys_link
| system_call
| __GI___link
| main
| __libc_start_main
|
--50.00%-- ext4_truncate
ext4_setattr
notify_change
do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] mb_mark_used
|
--- mb_mark_used
ext4_mb_use_best_found
ext4_mb_check_limits
ext4_mb_complex_scan_group
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] inode_change_ok
|
--- inode_change_ok
ext4_setattr
notify_change
|
|--50.00%-- do_truncate
| do_sys_ftruncate.constprop.13
| sys_ftruncate
| system_call
| __GI___ftruncate64
| main
| __libc_start_main
|
--50.00%-- chown_common
sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __block_write_begin
|
--- __block_write_begin
ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] sys_chmod
|
--- sys_chmod
__GI___chmod
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __alloc_pages_nodemask
|
--- __alloc_pages_nodemask
|
|--50.00%-- alloc_pages_current
| new_slab
| __slab_alloc
| kmem_cache_alloc
| ext4_alloc_inode
| alloc_inode
| new_inode_pseudo
| new_inode
| __ext4_new_inode
| ext4_mkdir
| vfs_mkdir
| sys_mkdir
| system_call
| __GI___mkdir
| main
| __libc_start_main
|
--50.00%-- __page_cache_alloc
grab_cache_page_write_begin
ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] unlock_two_nondirectories
|
--- unlock_two_nondirectories
|
|--50.00%-- SYSC_renameat
| sys_rename
| system_call
| rename
| main
| __libc_start_main
|
--50.00%-- vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.09% loader1 [kernel.kallsyms] [k] __d_instantiate
|
--- __d_instantiate
d_instantiate
ext4_link
vfs_link
sys_link
system_call
__GI___link
main
__libc_start_main
0.07% loader1 [kernel.kallsyms] [k] __printk_ratelimit
|
--- __printk_ratelimit
|
|--58.87%-- audit_log_lost
| audit_printk_skb
| audit_log_end
| audit_log_name
| audit_log_exit
| __audit_syscall_exit
| sysret_audit
| __GI___libc_chown
| main
| __libc_start_main
|
--41.13%-- audit_printk_skb
audit_log_end
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.07% perf [kernel.kallsyms] [k] generic_exec_single
|
--- generic_exec_single
smp_call_function_single
cpu_function_call
perf_event_enable
perf_event_for_each_child
perf_ioctl
do_vfs_ioctl
sys_ioctl
system_call
__GI___ioctl
cmd_record
run_builtin
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __module_address
|
--- __module_address
__module_text_address
is_module_text_address
func_ptr_is_kernel_text
notifier_call_chain
raw_notifier_call_chain
timekeeping_update.constprop.9
do_timer
tick_do_update_jiffies64
tick_sched_do_timer
tick_sched_timer
__run_hrtimer
hrtimer_interrupt
local_apic_timer_interrupt
smp_apic_timer_interrupt
apic_timer_interrupt
audit_log_vformat
audit_log_format
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] sys_symlink
|
--- sys_symlink
__GI___symlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] do_sys_open
|
--- do_sys_open
system_call
__GI___libc_open
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] dentry_free
|
--- dentry_free
__dentry_kill
dput
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __ext4_forget
|
--- __ext4_forget
ext4_free_blocks
ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] mem_cgroup_uncharge_start
|
--- mem_cgroup_uncharge_start
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] cap_inode_need_killpriv
|
--- cap_inode_need_killpriv
notify_change
chown_common
sys_lchown
system_call
__GI___lchown
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] dquot_alloc_inode
|
--- dquot_alloc_inode
__ext4_new_inode
ext4_symlink
vfs_symlink
sys_symlink
system_call
__GI___symlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] enqueue_entity
|
--- enqueue_entity
enqueue_task_fair
enqueue_task
activate_task
wake_up_new_task
do_fork
sys_clone
stub_clone
__libc_fork
create_load
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] up_write
|
--- up_write
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] security_file_open
|
--- security_file_open
vfs_open
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __d_move
|
--- __d_move
d_move
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 libc-2.19.so [.] __GI___libc_creat
|
--- __GI___libc_creat
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __ext4_ext_check
|
--- __ext4_ext_check
ext4_ext_truncate
ext4_truncate
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_mb_find_by_goal
|
--- ext4_mb_find_by_goal
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] putname
|
--- putname
user_path_at
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] jbd2_journal_begin_ordered_truncate
|
--- jbd2_journal_begin_ordered_truncate
ext4_setattr
notify_change
do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_free_blocks
|
--- ext4_free_blocks
ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] call_rcu_sched
|
--- call_rcu_sched
__dentry_kill
dput
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] generic_fillattr
|
--- generic_fillattr
vfs_getattr
vfs_fstatat
SYSC_newlstat
sys_newlstat
system_call
__lxstat64
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] security_path_link
|
--- security_path_link
system_call
__GI___link
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_init_acl
|
--- ext4_init_acl
__ext4_new_inode
ext4_symlink
vfs_symlink
sys_symlink
system_call
__GI___symlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] kfree_skbmem
|
--- kfree_skbmem
kfree_skb
audit_hold_skb
audit_printk_skb
audit_log_end
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___rmdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] mb_find_buddy
|
--- mb_find_buddy
mb_find_extent
ext4_mb_complex_scan_group
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] inode_sub_bytes
|
--- inode_sub_bytes
ext4_free_blocks
ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __d_alloc
|
--- __d_alloc
d_alloc
lookup_dcache
__lookup_hash
kern_path_create
user_path_create
sys_link
system_call
__GI___link
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] run_timer_softirq
|
--- run_timer_softirq
__do_softirq
irq_exit
smp_apic_timer_interrupt
apic_timer_interrupt
audit_log_vformat
audit_log_format
audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __srcu_read_lock
|
--- __srcu_read_lock
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_xattr_set_handle
|
--- ext4_xattr_set_handle
ext4_xattr_set
ext4_xattr_security_set
generic_removexattr
ima_inode_post_setattr
notify_change
chown_common
sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_free_inodes_count
|
--- ext4_free_inodes_count
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ima_match_policy
|
--- ima_match_policy
ima_get_action
process_measurement
ima_file_check
do_last
path_openat
do_filp_open
do_sys_open
sys_open
system_call
__GI___libc_open
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_da_invalidatepage
|
--- ext4_da_invalidatepage
truncate_inode_pages_range
truncate_pagecache
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_mb_use_preallocated
|
--- ext4_mb_use_preallocated
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] vfs_unlink
|
--- vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_inode_to_goal_block
|
--- ext4_inode_to_goal_block
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] generic_getxattr
|
--- generic_getxattr
get_vfs_caps_from_disk
audit_copy_inode
__audit_inode_child
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] release_pages
|
--- release_pages
__pagevec_release
truncate_inode_pages_range
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __init_rwsem
|
--- __init_rwsem
setup_object.isra.47
new_slab
__slab_alloc
kmem_cache_alloc
ext4_alloc_inode
alloc_inode
new_inode_pseudo
new_inode
__ext4_new_inode
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] integrity_iint_find
|
--- integrity_iint_find
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] jbd2__journal_start
|
--- jbd2__journal_start
__ext4_journal_start_sb
__ext4_new_inode
ext4_symlink
vfs_symlink
sys_symlink
system_call
__GI___symlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_rename
|
--- ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_ext_correct_indexes
|
--- ext4_ext_correct_indexes
ext4_ext_insert_extent
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] capable_wrt_inode_uidgid
|
--- capable_wrt_inode_uidgid
generic_permission
__inode_permission
inode_permission
may_open
do_last
path_openat
do_filp_open
do_sys_open
sys_open
system_call
__GI___libc_open
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] prepend.constprop.25
|
--- prepend.constprop.25
audit_log_d_path
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___chmod
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] mark_page_accessed
|
--- mark_page_accessed
__getblk
ext4_getblk
ext4_bread
__ext4_read_dirblock
ext4_add_entry
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_insert_dentry
|
--- ext4_insert_dentry
add_dirent_to_buf
ext4_add_entry
ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_mb_check_limits
|
--- ext4_mb_check_limits
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_es_remove_extent
|
--- ext4_es_remove_extent
truncate_inode_page
truncate_inode_pages_range
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] sys_close
|
--- sys_close
__GI___libc_close
create_load
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __kmalloc_track_caller
|
--- __kmalloc_track_caller
kstrdup
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] skb_free_head
|
--- skb_free_head
skb_release_data
skb_release_all
kfree_skb
audit_hold_skb
audit_printk_skb
audit_log_end
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__creat_nocancel
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_es_lookup_extent
|
--- ext4_es_lookup_extent
ext4_map_blocks
ext4_getblk
ext4_bread
__ext4_read_dirblock
ext4_add_entry
ext4_link
vfs_link
sys_link
system_call
__GI___link
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] mntput
|
--- mntput
__audit_syscall_exit
sysret_audit
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] __dentry_kill
|
--- __dentry_kill
dput
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] jbd2_journal_blocks_per_page
|
--- jbd2_journal_blocks_per_page
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] mb_find_extent
|
--- mb_find_extent
ext4_mb_regular_allocator
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] balance_dirty_pages_ratelimited
|
--- balance_dirty_pages_ratelimited
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] lookup_fast
|
--- lookup_fast
do_last
path_openat
do_filp_open
do_sys_open
sys_open
system_call
__GI___libc_open
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] dquot_initialize
|
--- dquot_initialize
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] ext4_get_inode_flags
|
--- ext4_get_inode_flags
ext4_mark_iloc_dirty
ext4_mark_inode_dirty
add_dirent_to_buf
ext4_add_entry
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] rw_verify_area
|
--- rw_verify_area
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] dnotify_flush
|
--- dnotify_flush
filp_close
__close_fd
sys_close
system_call
__GI___libc_close
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] may_delete
|
--- may_delete
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] vfs_rmdir
|
--- vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.05% loader1 [kernel.kallsyms] [k] unroll_tree_refs
|
--- unroll_tree_refs
sysret_audit
__GI___libc_write
main
__libc_start_main
0.05% rcu_sched [kernel.kallsyms] [k] lock_timer_base.isra.35
|
--- lock_timer_base.isra.35
rcu_gp_kthread
kthread
ret_from_fork
0.04% loader1 [kernel.kallsyms] [k] setattr_copy
|
--- setattr_copy
ext4_setattr
notify_change
chown_common
sys_lchown
system_call
__GI___lchown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] should_remove_suid
|
--- should_remove_suid
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] make_kgid
|
--- make_kgid
alloc_inode
new_inode_pseudo
new_inode
__ext4_new_inode
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] do_sync_write
|
--- do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __audit_getname
|
--- __audit_getname
user_path_parent
do_unlinkat
sys_unlink
system_call
__GI___unlink
create_load
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] skb_release_head_state
|
--- skb_release_head_state
skb_release_all
kfree_skb
audit_hold_skb
audit_printk_skb
audit_log_end
audit_log_exit
__audit_syscall_exit
sysret_audit
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] fsnotify_clear_marks_by_inode
|
--- fsnotify_clear_marks_by_inode
__fsnotify_inode_delete
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] mntget
|
--- mntget
__audit_getname
getname_flags
user_path_at_empty
sys_link
system_call
__GI___link
main
__libc_start_main
0.04% loader1 loader1 [.] create_load
|
--- create_load
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __call_rcu_nocb_enqueue
|
--- __call_rcu_nocb_enqueue
__call_rcu
call_rcu_sched
ext4_destroy_inode
destroy_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% rcu_sched [kernel.kallsyms] [k] update_stats_wait_end
|
--- update_stats_wait_end
set_next_entity
pick_next_task_fair
__schedule
schedule
schedule_timeout
rcu_gp_kthread
kthread
ret_from_fork
0.04% loader1 [kernel.kallsyms] [k] __alloc_fd
|
--- __alloc_fd
get_unused_fd_flags
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] kern_path_create
|
--- kern_path_create
user_path_create
sys_link
system_call
__GI___link
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_da_write_end
|
--- ext4_da_write_end
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] try_to_wake_up
|
--- try_to_wake_up
wake_up_process
process_timeout
call_timer_fn
run_timer_softirq
__do_softirq
irq_exit
smp_apic_timer_interrupt
apic_timer_interrupt
vsnprintf
audit_log_vformat
audit_log_format
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___unlink
create_load
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] dquot_drop
|
--- dquot_drop
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __fput
|
--- __fput
____fput
task_work_run
do_notify_resume
int_signal
__GI___libc_close
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_read_inode_bitmap
|
--- ext4_read_inode_bitmap
ext4_free_inode
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_ext_insert_extent
|
--- ext4_ext_insert_extent
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] chown_common
|
--- chown_common
sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] jbd2_journal_file_inode
|
--- jbd2_journal_file_inode
ext4_block_zero_page_range
ext4_block_truncate_page
ext4_truncate
ext4_setattr
notify_change
do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_used_dirs_count
|
--- ext4_used_dirs_count
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] generic_write_sync
|
--- generic_write_sync
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] task_work_run
|
--- task_work_run
do_notify_resume
int_signal
__GI___libc_close
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] put_pid
|
--- put_pid
____fput
task_work_run
do_notify_resume
int_signal
__GI___libc_close
main
__libc_start_main
0.04% loader1 libc-2.19.so [.] __GI___mkdir
|
--- __GI___mkdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] audit_comparator
|
--- audit_comparator
audit_filter_rules.isra.7
audit_filter_syscall
__audit_syscall_exit
sysret_audit
__GI___symlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] audit_compare_dname_path
|
--- audit_compare_dname_path
__audit_inode_child
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] sys_lchown
|
--- sys_lchown
__GI___lchown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] cap_task_getsecid
|
--- cap_task_getsecid
audit_log_task_context
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___fchmod
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] evm_inode_post_setattr
|
--- evm_inode_post_setattr
chown_common
sys_fchown
system_call
__GI___fchown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ____fput
|
--- ____fput
do_notify_resume
int_signal
__GI___libc_close
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] apparmor_path_chown
|
--- apparmor_path_chown
security_path_chown
chown_common
sys_chown
system_call
__GI___libc_chown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] tracesys
|
--- tracesys
__xstat64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] skb_release_all
|
--- skb_release_all
audit_hold_skb
audit_printk_skb
audit_log_end
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
__xstat64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] audit_log_d_path
|
--- audit_log_d_path
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___libc_chown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] _cond_resched
|
--- _cond_resched
ext4_reserve_inode_write
ext4_xattr_set_handle
ext4_xattr_set
ext4_xattr_security_set
generic_removexattr
ima_inode_post_setattr
notify_change
chmod_common
sys_fchmod
system_call
__GI___fchmod
main
__libc_start_main
0.04% loader1 libc-2.19.so [.] __lxstat64
|
--- __lxstat64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __srcu_read_unlock
|
--- __srcu_read_unlock
fsnotify
notify_change
do_truncate
do_sys_ftruncate.constprop.13
sys_ftruncate
system_call
__GI___ftruncate64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] timespec_trunc
|
--- timespec_trunc
ext4_setattr
notify_change
chown_common
sys_lchown
system_call
__GI___lchown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __slab_alloc
|
--- __slab_alloc
kmem_cache_alloc
ext4_alloc_inode
alloc_inode
new_inode_pseudo
new_inode
__ext4_new_inode
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] vfs_write
|
--- vfs_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] rb_erase
|
--- rb_erase
__es_remove_extent
ext4_es_remove_extent
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] inode_permission
|
--- inode_permission
link_path_walk
path_lookupat
filename_lookup
user_path_parent
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] res_counter_uncharge
|
--- res_counter_uncharge
truncate_inode_pages_range
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __d_drop
|
--- __d_drop
d_move
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] audit_log_untrustedstring
|
--- audit_log_untrustedstring
audit_log_task_info
audit_log_exit
__audit_syscall_exit
sysret_audit
__GI___libc_open
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] do_dentry_open
|
--- do_dentry_open
vfs_open
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __wake_up_common
|
--- __wake_up_common
__wake_up
jbd2_journal_stop
__ext4_journal_stop
ext4_rmdir
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] audit_serial
|
--- audit_serial
audit_log_start
audit_log_exit
__audit_syscall_exit
sysret_audit
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_inode_attach_jinode
|
--- ext4_inode_attach_jinode
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] copy_user_generic_string
|
--- copy_user_generic_string
SYSC_newlstat
sys_newlstat
system_call
__lxstat64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] sys_rename
|
--- sys_rename
rename
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] sys_link
|
--- sys_link
system_call
__GI___link
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_xattr_security_get
|
--- ext4_xattr_security_get
get_vfs_caps_from_disk
audit_copy_inode
__audit_inode
filename_lookup
user_path_parent
do_unlinkat
sys_unlink
system_call
__GI___unlink
create_load
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] cap_inode_rename
|
--- cap_inode_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_da_write_begin
|
--- ext4_da_write_begin
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] page_waitqueue
|
--- page_waitqueue
truncate_inode_pages_range
truncate_pagecache
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __jbd2_journal_temp_unlink_buffer
|
--- __jbd2_journal_temp_unlink_buffer
jbd2_journal_revoke
__ext4_forget
ext4_free_blocks
ext4_ext_remove_space
ext4_ext_truncate
ext4_truncate
ext4_evict_inode
evict
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] mnt_want_write
|
--- mnt_want_write
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_xattr_get
|
--- ext4_xattr_get
ext4_xattr_security_get
generic_getxattr
get_vfs_caps_from_disk
audit_copy_inode
__audit_inode_child
may_delete
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] cap_inode_getsecid
|
--- cap_inode_getsecid
audit_copy_inode
__audit_inode
filename_lookup
user_path_parent
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __dquot_alloc_space
|
--- __dquot_alloc_space
ext4_mb_new_blocks
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_find_entry
|
--- ext4_find_entry
ext4_unlink
vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] filename_lookup
|
--- filename_lookup
user_path_parent
do_unlinkat
sys_unlink
system_call
__GI___unlink
create_load
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_get_acl
|
--- ext4_get_acl
ext4_acl_chmod
ext4_setattr
notify_change
chmod_common
sys_fchmod
system_call
__GI___fchmod
main
__libc_start_main
0.04% loader1 libc-2.19.so [.] __GI___libc_chown
|
--- __GI___libc_chown
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] free_hot_cold_page
|
--- free_hot_cold_page
free_hot_cold_page_list
release_pages
__pagevec_release
truncate_inode_pages_range
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] inode_init_always
|
--- inode_init_always
alloc_inode
new_inode_pseudo
new_inode
__ext4_new_inode
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] dquot_active.isra.8
|
--- dquot_active.isra.8
dquot_initialize
ext4_rename
vfs_rename
SYSC_renameat
sys_rename
system_call
rename
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_set_inode_state
|
--- ext4_set_inode_state
ext4_symlink
vfs_symlink
sys_symlink
system_call
__GI___symlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __mem_cgroup_uncharge_common
|
--- __mem_cgroup_uncharge_common
mem_cgroup_uncharge_cache_page
delete_from_page_cache
truncate_inode_page
truncate_inode_pages_range
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_block_zero_page_range
|
--- ext4_block_zero_page_range
ext4_block_truncate_page
ext4_truncate
ext4_setattr
notify_change
do_truncate
vfs_truncate
do_sys_truncate
sys_truncate
system_call
truncate
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] d_lookup
|
--- d_lookup
lookup_dcache
__lookup_hash
kern_path_create
user_path_create
sys_link
system_call
__GI___link
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] find_get_pages
|
--- find_get_pages
truncate_inode_pages_range
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] radix_tree_lookup_element
|
--- radix_tree_lookup_element
radix_tree_lookup_slot
find_get_page
__find_get_block_slow
unmap_underlying_metadata
__block_write_begin
ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ihold
|
--- ihold
vfs_link
sys_link
system_call
__GI___link
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __audit_syscall_entry
|
--- __audit_syscall_entry
auditsys
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] get_empty_filp
|
--- get_empty_filp
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] task_tick_fair
|
--- task_tick_fair
scheduler_tick
update_process_times
tick_sched_handle.isra.17
tick_sched_timer
__run_hrtimer
hrtimer_interrupt
local_apic_timer_interrupt
smp_apic_timer_interrupt
apic_timer_interrupt
ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] audit_log_fcaps
|
--- audit_log_fcaps
audit_log_name
audit_log_exit
__audit_syscall_exit
sysret_audit
rename
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] security_path_symlink
|
--- security_path_symlink
system_call
__GI___symlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_init_security
|
--- ext4_init_security
ext4_create
vfs_create
do_last
path_openat
do_filp_open
do_sys_open
sys_creat
system_call
__creat_nocancel
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] inode_wait_for_writeback
|
--- inode_wait_for_writeback
iput
d_delete
vfs_rmdir
do_rmdir
sys_rmdir
system_call
__GI___rmdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] __d_lookup
|
--- __d_lookup
d_lookup
lookup_dcache
__lookup_hash
do_unlinkat
sys_unlink
system_call
__GI___unlink
create_load
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] inode_add_rsv_space
|
--- inode_add_rsv_space
__dquot_alloc_space
ext4_da_get_block_prep
__block_write_begin
ext4_da_write_begin
generic_file_buffered_write
__generic_file_aio_write
generic_file_aio_write
ext4_file_write
do_sync_write
vfs_write
sys_write
system_call
__GI___libc_write
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_mb_regular_allocator
|
--- ext4_mb_regular_allocator
ext4_ext_map_blocks
ext4_map_blocks
ext4_getblk
ext4_bread
ext4_append
ext4_mkdir
vfs_mkdir
sys_mkdir
system_call
__GI___mkdir
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] ext4_get_inode_loc
|
--- ext4_get_inode_loc
ext4_mark_inode_dirty
ext4_unlink
vfs_unlink
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] _copy_to_user
|
--- _copy_to_user
SYSC_newlstat
sys_newlstat
system_call
__lxstat64
main
__libc_start_main
0.04% loader1 [kernel.kallsyms] [k] fput
|
--- fput
filp_close
__close_fd
sys_close
system_call
__GI___libc_close
main
__libc_start_main
0.02% swapper [kernel.kallsyms] [k] cpu_startup_entry
|
--- cpu_startup_entry
start_secondary
0.02% kworker/5:1 [kernel.kallsyms] [k] find_next_bit
|
--- find_next_bit
od_dbs_timer
process_one_work
worker_thread
kthread
ret_from_fork
0.01% java [kernel.kallsyms] [k] idle_balance
|
--- idle_balance
__schedule
schedule
futex_wait_queue_me
futex_wait
do_futex
sys_futex
system_call
pthread_cond_timedwait@@GLIBC_2.3.2
_ZN7Monitor5IWaitEP6Threadl
_ZN7Monitor4waitEblb
_ZNK13WatcherThread5sleepEv
_ZN13WatcherThread3runEv
_ZL10java_startP6Thread
start_thread
0.01% swapper [kernel.kallsyms] [k] this_cpu_load
|
--- this_cpu_load
menu_select
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.01% swapper [kernel.kallsyms] [k] rcu_idle_exit
|
--- rcu_idle_exit
|
|--79.96%-- cpu_startup_entry
| start_secondary
|
--20.04%-- start_secondary
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] __slab_free
|
--- __slab_free
kmem_cache_free
jbd2_journal_write_revoke_records
jbd2_journal_commit_transaction
kjournald2
kthread
ret_from_fork
0.01% swapper [kernel.kallsyms] [k] native_write_msr_safe
|
--- native_write_msr_safe
intel_pmu_enable_all
intel_pmu_nhm_enable_all
x86_pmu_enable
perf_pmu_enable
x86_pmu_commit_txn
group_sched_in
__perf_event_enable
remote_function
generic_smp_call_function_single_interrupt
smp_call_function_single_interrupt
call_function_single_interrupt
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
|
|--98.06%-- start_secondary
|
--1.94%-- rest_init
start_kernel
x86_64_start_reservations
x86_64_start_kernel
0.01% kworker/1:0 [kernel.kallsyms] [k] pci_conf1_read
|
--- pci_conf1_read
pci_read
pci_bus_read_config_dword
be_detect_error
be_func_recovery_task
process_one_work
worker_thread
kthread
ret_from_fork
0.01% swapper [kernel.kallsyms] [k] read_tsc
|
--- read_tsc
cpuidle_enter_state
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] __find_get_block
|
--- __find_get_block
jbd2_clear_buffer_revoked_flags
jbd2_journal_commit_transaction
kjournald2
kthread
ret_from_fork
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] jbd2_journal_write_revoke_records
|
--- jbd2_journal_write_revoke_records
jbd2_journal_commit_transaction
kjournald2
kthread
ret_from_fork
0.01% swapper [kernel.kallsyms] [k] cpuidle_idle_call
|
--- cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.01% kworker/11:1 [kernel.kallsyms] [k] idle_balance
|
--- idle_balance
__schedule
schedule
worker_thread
kthread
ret_from_fork
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] put_page
|
--- put_page
__find_get_block_slow
__find_get_block
jbd2_clear_buffer_revoked_flags
jbd2_journal_commit_transaction
kjournald2
kthread
ret_from_fork
0.01% jbd2/sdb1-8 [kernel.kallsyms] [k] jbd2_clear_buffer_revoked_flags
|
--- jbd2_clear_buffer_revoked_flags
jbd2_journal_commit_transaction
kjournald2
kthread
ret_from_fork
0.01% java libjvm.so [.] _ZN7Monitor4waitEblb
|
--- _ZN7Monitor4waitEblb
_ZN8VMThread4loopEv
_ZN8VMThread3runEv
_ZL10java_startP6Thread
start_thread
0.00% swapper [kernel.kallsyms] [k] hrtimer_interrupt
|
--- hrtimer_interrupt
local_apic_timer_interrupt
smp_apic_timer_interrupt
apic_timer_interrupt
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.00% swapper [kernel.kallsyms] [k] int_sqrt
|
--- int_sqrt
menu_select
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.00% swapper [kernel.kallsyms] [k] rcu_sysidle_enter
|
--- rcu_sysidle_enter
rcu_idle_enter
cpu_startup_entry
start_secondary
0.00% swapper [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
|
--- _raw_spin_unlock_irqrestore
__hrtimer_start_range_ns
hrtimer_start
tick_nohz_stop_sched_tick
__tick_nohz_idle_enter
tick_nohz_idle_enter
cpu_startup_entry
start_secondary
0.00% loader1 [kernel.kallsyms] [k] audit_hold_skb
|
--- audit_hold_skb
audit_printk_skb
audit_log_end
audit_log_exit
__audit_syscall_exit
sysret_audit
__xstat64
main
__libc_start_main
0.00% ovsdb-server ovsdb-server [.] 0x000000000004c579
|
--- 0x44c579
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0x15bc030
0x15aabd0
0.00% swapper [kernel.kallsyms] [k] hrtimer_forward
|
--- hrtimer_forward
tick_nohz_restart
tick_nohz_idle_exit
cpu_startup_entry
start_secondary
0.00% swapper [kernel.kallsyms] [k] tick_program_event
|
--- tick_program_event
local_apic_timer_interrupt
smp_apic_timer_interrupt
apic_timer_interrupt
cpuidle_idle_call
arch_cpu_idle
cpu_startup_entry
start_secondary
0.00% perf [kernel.kallsyms] [k] native_write_msr_safe
|
--- native_write_msr_safe
intel_pmu_enable_all
intel_pmu_nhm_enable_all
x86_pmu_enable
perf_pmu_enable
x86_pmu_commit_txn
group_sched_in
__perf_event_enable
remote_function
smp_call_function_single
cpu_function_call
perf_event_enable
perf_event_for_each_child
perf_ioctl
do_vfs_ioctl
sys_ioctl
system_call
__GI___ioctl
cmd_record
run_builtin
main
__libc_start_main
0.00% loader1 [kernel.kallsyms] [k] native_write_msr_safe
|
--- native_write_msr_safe
intel_pmu_enable_all
intel_pmu_nhm_enable_all
x86_pmu_enable
perf_pmu_enable
x86_pmu_commit_txn
group_sched_in
__perf_event_enable
remote_function
generic_smp_call_function_single_interrupt
smp_call_function_single_interrupt
call_function_single_interrupt
truncate_inode_page
truncate_inode_pages_range
truncate_inode_pages
ext4_evict_inode
evict
iput
do_unlinkat
sys_unlink
system_call
__GI___unlink
main
__libc_start_main
#
# (For a higher level overview, try: perf report --sort comm,dso)
#
[-- Attachment #4: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply
* [GIT PULL] Audit fixes for 3.20
From: Paul Moore @ 2015-02-11 19:36 UTC (permalink / raw)
To: Linus Torvalds; +Cc: linux-audit, linux-kernel
Hi Linus,
Just one patch from the audit tree for v3.20, and a very minor one at that.
The patch simply removes an old, unused field from the audit_krule structure,
a private audit-only struct. In audit related news, we did a proper overhaul
of the audit pathname code and removed the nasty getname()/putname() hacks for
audit, you should see those patches in Al's vfs tree if you haven't already.
That's it for audit this time, let's hope for a quiet -rcX series.
Thanks,
-Paul
---
The following changes since commit fcf22d8267ad2601fe9b6c549d1be96401c23e0b:
audit: create private file name copies when auditing inodes
(2014-12-30 09:26:21 -0500)
are available in the git repository at:
git://git.infradead.org/users/pcmoore/audit upstream
for you to fetch changes up to 2fded7f44b8fcf79e274c3f0cfbd0298f95308f3:
audit: remove vestiges of vers_ops (2015-01-20 10:48:32 -0500)
----------------------------------------------------------------
Richard Guy Briggs (1):
audit: remove vestiges of vers_ops
include/linux/audit.h | 1 -
kernel/auditfilter.c | 2 --
2 files changed, 3 deletions(-)
^ permalink raw reply
* Re: Linux audit performance impact
From: Richard Guy Briggs @ 2015-02-11 16:51 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL); +Cc: linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914165BA8B@G4W3222.americas.hpqcorp.net>
On 15/02/06, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Hi all,
>
> Please find the below the details of the performance test we ran.
> It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
>
> Kernel Version:
> root > uname -r
> 3.13.0-36-generic
>
> OS Version:
> Ubuntu 14.04.1
>
> No. of CPUs:
> root > nproc
> 24
>
> Audit Status:
> root > auditctl -s
> AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320 lost=57190353 backlog=0
>
> Rules Configured:
> root > auditctl -l
> LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
>
> Attached is the program used to load the system.
>
> Results:
>
> Without enabling audit 12.29
> With auditing enabled and no rules configured 12.31
> With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
This would be more meaningful if you hacked the kernel to drain the
queue figuratively to /dev/nul to eliminate the effect of auditd
draining it, or syslog covering for a missing auditd. This stat doesn't
tell us that much since the I/O act can vary significantly per
installation. That one rule you chose is pretty unnaturally abusive and
needs to be carefully thought out to avoid self-measurement.
> The degradation is around 200%
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 04, 2015 9:46 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > The intent is to calculate the performance impact by the auditing
> > components such as
> >
> > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
>
> Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
>
> > 2) impact because of running auditd - log format NOLOG
> > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Tuesday, February 03, 2015 10:33 PM
> > To: Satish Chandra Kilaru
> > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/03, Satish Chandra Kilaru wrote:
> > > Thanks for The info. But my question was rhetorical... I meant to
> > > say that it would not be much... She is trying to bombard the system
> > > with open calls ... So lots and lots of events will be generated and
> > > kernel has to write down the events some where or discard them...
> >
> > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> >
> > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > >
> > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > How many events can kernel accumulate without I/o ?
> > > >
> > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > set it to 320. It is now possible to set it to "0" which means
> > > > limited only by system resources. See "man auditctl", "-b"
> > > > option. An event can be made up of several buffers.
> > > >
> > > > Of course, how long a system lasts before the queue blows up
> > > > depends on your rule set...
> > > >
> > > > However, at the moment, it will still write out to klog if auditd
> > > > isn't running.
> > > >
> > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU OSTL)
> > > > > < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > >
> > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > collection),
> > > > > > but just do not want the records to delivered to user space
> > > > > > since I
> > > > want to
> > > > > > remove the I/O overhead while running the performance test.
> > > > > > Is there any option for this?
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com <javascript:;>
> > > > <javascript:;>]
> > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;>
> > > > > > Subject: Re: Linux audit performance impact
> > > > > >
> > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > Please read my question as “Is there any option to configure
> > > > > > > kaudit not to log audit records to syslog? when auditd not running.”
> > > > > >
> > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > audit=0 in
> > > > its
> > > > > > place. This will stop all but AVCs and if auditd has ever run
> > > > > > since
> > > > boot.
> > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > to run
> > > > auditd.
> > > > > >
> > > > > > There is a feature request that is likely coming soon that
> > > > > > could be
> > > > > > useful:
> > > > > >
> > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > >
> > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > Subject: RE: Linux audit performance impact
> > > > > > >
> > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > records to
> > > > > > syslog when auditd is running?
> > > > > > > This way we can assess the impact of enabling audit without
> > > > > > > involving
> > > > > > disk I/o overhead.
> > > > > > >
> > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > <javascript:;> <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > To: Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > linux-audit@redhat.com <javascript:;>
> > > > > > <javascript:;>>; Viswanath,
> > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > I agree with you... but writing to disk can trigger further
> > > > > > > events
> > > > > > leading spiralling of events...
> > > > > > > I brought down my server few times with stupid rules...
> > > > > > >
> > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > <sgrubb@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > <javascript:;>>> wrote:
> > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > Kilaru
> > > > wrote:
> > > > > > > > Write your own program to receive audit events directly
> > > > > > > > without using auditd...
> > > > > > > > That should be faster ....
> > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > >
> > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > if you
> > > > want
> > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > made it to disk and the priority of that audit daemon. Then
> > > > > > > you also have all
> > > > the
> > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > for any disk performance critical system.
> > > > > > >
> > > > > > > -Steve
> > > > > > >
> > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > (MCOU
> > > > > > > > OSTL)
> > > > <
> > > > > > > >
> > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > logeswari.pv@hp.com <javascript:;>
> > > > > > <javascript:;>>> wrote:
> > > > > > > > > Hi Steve,
> > > > > > > > >
> > > > > > > > > I am Logeswari working for HP.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > Suse linux
> > > > to
> > > > > > > > > help us evaluate linux audit as data source for our host
> > > > > > > > > based
> > > > IDS.
> > > > > > > > >
> > > > > > > > > When we ran our own performance test with a test audispd
> > > > > > > > > plugin, we found if a system can perform 200000
> > > > > > > > > open/close system calls per second without auditing,
> > > > > > > > > system can perform only 3000 open/close system calls
> > > > > > > > > auditing is enabled for open/close system call which is
> > > > > > > > > a HUGE impact on the system performance. It would
> > > > be
> > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > reason
> > > > > > > > > behind it and can we fix it?
> > > > > > > > >
> > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > impact? If
> > > > > > yes,
> > > > > > > > > can you please share the numbers and also the
> > > > > > > > > steps/programs used the run the same.
> > > > > > > > >
> > > > > > > > > 3) Help us validating the performance test we have done in
> > > > our
> > > > > > test
> > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > close system
> > > > > > calls.
> > > > > > > > >
> > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > >
> > > > > > > > > We used time command to determine how much time the
> > > > > > > > > system took
> > > > to
> > > > > > > > > complete 50000 open/close system calls without (results
> > > > > > > > > attached
> > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > With-auditing-RAW)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > System details:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1 CPU machine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *OS Version*
> > > > > > > > >
> > > > > > > > > RHEL 6.5
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *Kernel Version*
> > > > > > > > >
> > > > > > > > > uname –r
> > > > > > > > >
> > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > for most
> > > > of
> > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks & Regards,
> > > > > > > > >
> > > > > > > > > Logeswari.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Please Donate to www.wikipedia.org<http://www.wikipedia.org>
> > > > > >
> > > > > > > --
> > > > > > > Linux-audit mailing list
> > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > >
> > > > > >
> > > > > > - RGB
> > > > > >
> > > > > > --
> > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > Canada
> > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > +1.613.693.0684x3545
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Please Donate to www.wikipedia.org
> > > >
> > > > - RGB
> > > >
> > > > --
> > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > +1.613.693.0684x3545
> > > >
> > >
> > >
> > > --
> > > Please Donate to www.wikipedia.org
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
> #include <stdio.h>
> #include <stdlib.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <unistd.h>
> #include <errno.h>
>
> void create_load(int iters);
> void cleanup();
>
> int high_rate = 0;
> int num_iters = 100000;
> int fd1;
> char file1[50];
> char file2[50];
> char dir1[50];
> char symlink1[50];
>
> /* Purpose: To create system load by invoking system calls used by templates.
> *
> * Note: The unlink(2) of a file can be an expensive operation (i.e., event
> * rate goes way down).
> */
>
> main(int argc, char **argv) {
>
> int num_children=1;
> int iters;
> int i;
> char c;
>
> while ((c = getopt(argc, argv, "hi:")) != -1) {
> switch (c) {
> case 'h':
> /*
> * Desire "high" event rate
> */
> high_rate = 1;
> argc--;
> break;
> case 'i':
> /*
> * Desire a specified number of iterations
> */
> num_iters = atoi(optarg);
> argc--;
> break;
> default:
> fprintf(stderr,"Unknown option: %c\n",optarg);
> exit(1);
> }
> }
>
>
> /*if(argv[optind] != NULL) {
> num_children = atoi(argv[optind]);
> } else {
> num_children = 0;
> }
> Register cleanup routine */
> fprintf(stderr,"Registering cleanup routine...\n");
> if (atexit(cleanup) == -1) {
> fprintf(stderr,"Error calling atexit(), errno=%d(%s)\n",
> errno,strerror(errno));
> exit(1);
> }
>
>
> /* fork child processes, if any requested */
> for(i=1; i < num_children; i++) {
> if(fork() == 0) {
>
> printf("child pid: %d\n",getpid());
>
> /* Setup file names based on child's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> /* each child creates load */
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
> }
> }
>
> /* Parent creates load also */
> printf("parent pid: %d\n",getpid());
>
> /* Setup file names based on parent's pid */
> sprintf(file1,"./file1_%d",getpid());
> sprintf(file2,"./file2_%d",getpid());
> sprintf(dir1,"./dir1_%d",getpid());
> sprintf(symlink1,"./file1symlink_%d",getpid());
>
> iters=0;
> if (num_iters == -1) {
> while(1) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> } else {
> while(iters < num_iters) {
> create_load(iters);
> iters++;
> if( (iters % 1000) == 0) {
> printf("pid %d iteration %d\n",getpid(),iters);
> }
> }
> }
>
> } /* main */
>
>
> void create_load(int iters) {
>
> int pid;
> char *args[2];
> struct stat stat_buf;
>
> fd1 = creat(file1,0x644);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: creat() returned error for file %s, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> fd1 = open(file1, O_RDWR, 0777);
> if (fd1 == -1) {
> fprintf(stderr,"pid %d: open() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Chown this file to root instead of user ids so that we don't generate a
> * non-owned alert when the file is truncated when invoking creat() again
> * as root on an existing file owned by another user.
> */
> if (chown(file1,0,0) == -1) {
> fprintf(stderr,"pid %d: chown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (fchown(fd1,0,0) == -1) {
> fprintf(stderr,"pid %d: fchown(%d,%d) returned error, errno=%d(%s)\n",
> getpid(),0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (chmod(file1, S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) == -1) {
> fprintf(stderr,"pid %d: chmod(S_IRUSR|S_IWUSR|S_IRGRP|S_IROTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (fchmod(fd1, S_IXUSR|S_IXGRP|S_IXOTH) == -1) {
> fprintf(stderr,"pid %d: fchmod(S_IXUSR|S_IXGRP|S_IXOTH) returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
>
> if (write(fd1,"Some stuff",strlen("Some stuff")) == -1) {
> fprintf(stderr,"pid %d: write() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (ftruncate(fd1,7) == -1) {
> fprintf(stderr,"pid %d: ftruncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (close(fd1) == -1) {
> fprintf(stderr,"pid %d: close() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> if (truncate(file1,3) == -1) {
> fprintf(stderr,"pid %d: truncate() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rename(file1,file2) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (rename(file2,file1) == -1) {
> fprintf(stderr,"pid %d: rename(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file2,file1,errno,strerror(errno));
> exit(1);
> }
> if (link(file1,file2) == -1) {
> fprintf(stderr,"pid %d: link(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,file2,errno,strerror(errno));
> exit(1);
> }
> if (symlink(file1,symlink1) == -1) {
> fprintf(stderr,"pid %d: symlink(%s,%s) returned error, errno=%d(%s)\n",
> getpid(),file1,symlink1,errno,strerror(errno));
> exit(1);
> }
> if (lchown(symlink1,0,0) == -1) {
> fprintf(stderr,"pid %d: lchown(%s,%d,%d) returned error, errno=%d(%s)\n",
> getpid(),symlink1,0,0,errno,strerror(errno));
> exit(1);
> }
>
> if (lstat(symlink1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: lstat(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (stat(file1,&stat_buf) == -1) {
> fprintf(stderr,"pid %d: stat(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file1,errno,strerror(errno));
> exit(1);
> }
> if (unlink(file2) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),file2,errno,strerror(errno));
> exit(1);
> }
> if (unlink(symlink1) == -1) {
> fprintf(stderr,"pid %d: unlink(%s) returned error, errno=%d(%s)\n",
> getpid(),symlink1,errno,strerror(errno));
> exit(1);
> }
> if (mkdir(dir1,S_IRUSR|S_IWUSR|S_IXUSR|S_IRGRP|S_IXGRP) == -1) {
> fprintf(stderr,"pid %d: mkdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> if (rmdir(dir1) == -1) {
> fprintf(stderr,"pid %d: rmdir() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
>
> /* Fork every 10000 iterations to not use up process resources too quickly */
> if ( (iters % 10000) == 0) {
> pid = fork();
> if(pid == 0) {
> fprintf(stderr,"child pid %d: fork!\n",getpid());
> // child
> args[0] = "/bin/ls";
> args[1] = NULL;
> close(1);
> close(2);
> execve(args[0], args, NULL);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: fork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> } else {
> fprintf(stderr,"parent pid %d, child pid: %d: fork!\n",getpid(),pid);
> }
>
> pid = vfork();
> if(pid == 0) {
> args[0] = "/bin/pwd";
> args[1] = NULL;
> close(1);
> close(2);
> execv(args[0], args);
> fprintf(stderr,"pid %d: execve(%s) returned error, errno=%d(%s)\n",
> getpid(),args[0],errno,strerror(errno));
> _exit(1);
> } else if (pid < 0) {
> fprintf(stderr,"pid %d: vfork() returned error, errno=%d(%s)\n",
> getpid(),errno,strerror(errno));
> exit(1);
> }
> }
>
> /* Make sure everything is cleaned up and deleted before returning */
> cleanup();
>
> } /* create_load() */
>
> void cleanup() {
> close(fd1);
> unlink(file1);
> unlink(file2);
> unlink(symlink1);
> unlink(dir1);
> return;
> }
> --
> Linux-audit mailing list
> Linux-audit@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-audit
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* Re: Linux audit performance impact
From: Richard Guy Briggs @ 2015-02-11 16:45 UTC (permalink / raw)
To: Viswanath, Logeswari P (MCOU OSTL); +Cc: linux-audit@redhat.com
In-Reply-To: <9DBA79E0CE64AA42B07DEDAAD0F7DB914165C222@G4W3222.americas.hpqcorp.net>
On 15/02/11, Viswanath, Logeswari P (MCOU OSTL) wrote:
> Another question, why was it decided to have multiple records per audit event?
I seem to recall it was to be able to filter unneeded information to
speed up processing. It does generate more, but some types of searches
can benefit from avoiding to have to parse records in which it has no
interest.
> For eg:
>
> type=SYSCALL msg=audit(1420988184.991:65696718): arch=c000003e syscall=2 success=yes exit=3 a0=e9f400 a1=0 a2=0 a3=5 items=1 ppid=2934 pid=2956 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=947 comm="vi" exe="/bin/vi" key=(null)
> type=CWD msg=audit(1420988184.991:65696718): cwd="/root/ids/bkp"
> type=PATH msg=audit(1420988184.991:65696718): item=0 name="../loader.c" inode=1862956 dev=08:02 mode=0100777 ouid=0 ogid=0 rdev=00:00
>
> Also, it would be great if one can help me answering my questions in the below mail?
>
> -----Original Message-----
> From: Viswanath, Logeswari P (MCOU OSTL)
> Sent: Friday, February 06, 2015 5:23 PM
> To: 'Richard Guy Briggs'
> Cc: 'Satish Chandra Kilaru'; 'Steve Grubb'; 'linux-audit@redhat.com'
> Subject: RE: Linux audit performance impact
>
> One more question, I have enabled all system calls for auditing and auditd is not running.
> Will printk result in write system call which in turn be audited?
> If yes, is there any way to ignore auditing for a specific processes
> such as syslogd to avoid auditing these extra write system calls?
Pre-pend a rule to exclude the activity of syslog by PID...
> -----Original Message-----
> From: Viswanath, Logeswari P (MCOU OSTL)
> Sent: Friday, February 06, 2015 12:17 PM
> To: 'Richard Guy Briggs'
> Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> Subject: RE: Linux audit performance impact
>
> Hi all,
>
> Please find the below the details of the performance test we ran.
> It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
>
> Kernel Version:
> root > uname -r
> 3.13.0-36-generic
>
> OS Version:
> Ubuntu 14.04.1
>
> No. of CPUs:
> root > nproc
> 24
>
> Audit Status:
> root > auditctl -s
> AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320 lost=57190353 backlog=0
>
> Rules Configured:
> root > auditctl -l
> LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
>
> Attached is the program used to load the system.
>
> Results:
>
> Without enabling audit 12.29
> With auditing enabled and no rules configured 12.31
> With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
>
> The degradation is around 200%
>
> Regards,
> Logeswari.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Wednesday, February 04, 2015 9:46 PM
> To: Viswanath, Logeswari P (MCOU OSTL)
> Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > The intent is to calculate the performance impact by the auditing
> > components such as
> >
> > 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
>
> Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
>
> > 2) impact because of running auditd - log format NOLOG
> > 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
> >
> > -----Original Message-----
> > From: Richard Guy Briggs [mailto:rgb@redhat.com]
> > Sent: Tuesday, February 03, 2015 10:33 PM
> > To: Satish Chandra Kilaru
> > Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> > linux-audit@redhat.com
> > Subject: Re: Linux audit performance impact
> >
> > On 15/02/03, Satish Chandra Kilaru wrote:
> > > Thanks for The info. But my question was rhetorical... I meant to
> > > say that it would not be much... She is trying to bombard the system
> > > with open calls ... So lots and lots of events will be generated and
> > > kernel has to write down the events some where or discard them...
> >
> > Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
> >
> > > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> > >
> > > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > > How many events can kernel accumulate without I/o ?
> > > >
> > > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > > set it to 320. It is now possible to set it to "0" which means
> > > > limited only by system resources. See "man auditctl", "-b"
> > > > option. An event can be made up of several buffers.
> > > >
> > > > Of course, how long a system lasts before the queue blows up
> > > > depends on your rule set...
> > > >
> > > > However, at the moment, it will still write out to klog if auditd
> > > > isn't running.
> > > >
> > > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU OSTL)
> > > > > < logeswari.pv@hp.com <javascript:;>> wrote:
> > > > >
> > > > > > I don't want to disable auditing (i.e. disable audit record
> > > > collection),
> > > > > > but just do not want the records to delivered to user space
> > > > > > since I
> > > > want to
> > > > > > remove the I/O overhead while running the performance test.
> > > > > > Is there any option for this?
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com <javascript:;>
> > > > <javascript:;>]
> > > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;>
> > > > > > Subject: Re: Linux audit performance impact
> > > > > >
> > > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > > Please read my question as “Is there any option to configure
> > > > > > > kaudit not to log audit records to syslog? when auditd not running.”
> > > > > >
> > > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > > audit=0 in
> > > > its
> > > > > > place. This will stop all but AVCs and if auditd has ever run
> > > > > > since
> > > > boot.
> > > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > > to run
> > > > auditd.
> > > > > >
> > > > > > There is a feature request that is likely coming soon that
> > > > > > could be
> > > > > > useful:
> > > > > >
> > > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > > "If no audit daemon is running, but an audit multicast
> > > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > > >
> > > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > Subject: RE: Linux audit performance impact
> > > > > > >
> > > > > > > Is there any option to configure kaudit not to log audit
> > > > > > > records to
> > > > > > syslog when auditd is running?
> > > > > > > This way we can assess the impact of enabling audit without
> > > > > > > involving
> > > > > > disk I/o overhead.
> > > > > > >
> > > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > > <javascript:;> <javascript:;>]
> > > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > > To: Steve Grubb
> > > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > > linux-audit@redhat.com <javascript:;>
> > > > > > <javascript:;>>; Viswanath,
> > > > > > > Logeswari P (MCOU OSTL)
> > > > > > > Subject: Re: Linux audit performance impact
> > > > > > >
> > > > > > > I agree with you... but writing to disk can trigger further
> > > > > > > events
> > > > > > leading spiralling of events...
> > > > > > > I brought down my server few times with stupid rules...
> > > > > > >
> > > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > > <sgrubb@redhat.com
> > > > <javascript:;>
> > > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > > <javascript:;>>> wrote:
> > > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > > Kilaru
> > > > wrote:
> > > > > > > > Write your own program to receive audit events directly
> > > > > > > > without using auditd...
> > > > > > > > That should be faster ....
> > > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > > >
> > > > > > > But even that is configurable in many ways. You can decide
> > > > > > > if you
> > > > want
> > > > > > > logging to disk or not and what kind of assurance that it
> > > > > > > made it to disk and the priority of that audit daemon. Then
> > > > > > > you also have all
> > > > the
> > > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > > for any disk performance critical system.
> > > > > > >
> > > > > > > -Steve
> > > > > > >
> > > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > > (MCOU
> > > > > > > > OSTL)
> > > > <
> > > > > > > >
> > > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > > logeswari.pv@hp.com <javascript:;>
> > > > > > <javascript:;>>> wrote:
> > > > > > > > > Hi Steve,
> > > > > > > > >
> > > > > > > > > I am Logeswari working for HP.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > > Suse linux
> > > > to
> > > > > > > > > help us evaluate linux audit as data source for our host
> > > > > > > > > based
> > > > IDS.
> > > > > > > > >
> > > > > > > > > When we ran our own performance test with a test audispd
> > > > > > > > > plugin, we found if a system can perform 200000
> > > > > > > > > open/close system calls per second without auditing,
> > > > > > > > > system can perform only 3000 open/close system calls
> > > > > > > > > auditing is enabled for open/close system call which is
> > > > > > > > > a HUGE impact on the system performance. It would
> > > > be
> > > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > > reason
> > > > > > > > > behind it and can we fix it?
> > > > > > > > >
> > > > > > > > > 2) Have anyone done any benchmarking for performance
> > > > impact? If
> > > > > > yes,
> > > > > > > > > can you please share the numbers and also the
> > > > > > > > > steps/programs used the run the same.
> > > > > > > > >
> > > > > > > > > 3) Help us validating the performance test we have done in
> > > > our
> > > > > > test
> > > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > > close system
> > > > > > calls.
> > > > > > > > >
> > > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > > >
> > > > > > > > > We used time command to determine how much time the
> > > > > > > > > system took
> > > > to
> > > > > > > > > complete 50000 open/close system calls without (results
> > > > > > > > > attached
> > > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > > With-auditing-RAW)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > System details:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 1 CPU machine
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *OS Version*
> > > > > > > > >
> > > > > > > > > RHEL 6.5
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *Kernel Version*
> > > > > > > > >
> > > > > > > > > uname –r
> > > > > > > > >
> > > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > > for most
> > > > of
> > > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thanks & Regards,
> > > > > > > > >
> > > > > > > > > Logeswari.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Please Donate to www.wikipedia.org<http://www.wikipedia.org>
> > > > > >
> > > > > > > --
> > > > > > > Linux-audit mailing list
> > > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > > >
> > > > > >
> > > > > > - RGB
> > > > > >
> > > > > > --
> > > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > > Canada
> > > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > > +1.613.693.0684x3545
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Please Donate to www.wikipedia.org
> > > >
> > > > - RGB
> > > >
> > > > --
> > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > > Systems, Red Hat Remote, Ottawa, Canada
> > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > +1.613.693.0684x3545
> > > >
> > >
> > >
> > > --
> > > Please Donate to www.wikipedia.org
> >
> > - RGB
> >
> > --
> > Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> > Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> > Ottawa, Canada
> > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > +1.613.693.0684x3545
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com>
> Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat
Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* RE: Linux audit performance impact
From: Viswanath, Logeswari P (MCOU OSTL) @ 2015-02-11 14:16 UTC (permalink / raw)
To: Richard Guy Briggs; +Cc: linux-audit@redhat.com
In-Reply-To: <20150204161557.GF18752@madcap2.tricolour.ca>
Another question, why was it decided to have multiple records per audit event?
For eg:
type=SYSCALL msg=audit(1420988184.991:65696718): arch=c000003e syscall=2 success=yes exit=3 a0=e9f400 a1=0 a2=0 a3=5 items=1 ppid=2934 pid=2956 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts3 ses=947 comm="vi" exe="/bin/vi" key=(null)
type=CWD msg=audit(1420988184.991:65696718): cwd="/root/ids/bkp"
type=PATH msg=audit(1420988184.991:65696718): item=0 name="../loader.c" inode=1862956 dev=08:02 mode=0100777 ouid=0 ogid=0 rdev=00:00
Also, it would be great if one can help me answering my questions in the below mail?
-----Original Message-----
From: Viswanath, Logeswari P (MCOU OSTL)
Sent: Friday, February 06, 2015 5:23 PM
To: 'Richard Guy Briggs'
Cc: 'Satish Chandra Kilaru'; 'Steve Grubb'; 'linux-audit@redhat.com'
Subject: RE: Linux audit performance impact
One more question, I have enabled all system calls for auditing and auditd is not running.
Will printk result in write system call which in turn be audited?
If yes, is there any way to ignore auditing for a specific processes such as syslogd to avoid auditing these extra write system calls?
-----Original Message-----
From: Viswanath, Logeswari P (MCOU OSTL)
Sent: Friday, February 06, 2015 12:17 PM
To: 'Richard Guy Briggs'
Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
Subject: RE: Linux audit performance impact
Hi all,
Please find the below the details of the performance test we ran.
It would be great if we get help to identify the reason behind the degradation and the ways of improving it.
Kernel Version:
root > uname -r
3.13.0-36-generic
OS Version:
Ubuntu 14.04.1
No. of CPUs:
root > nproc
24
Audit Status:
root > auditctl -s
AUDIT_STATUS: enabled=1 flag=1 pid=0 rate_limit=0 backlog_limit=320 lost=57190353 backlog=0
Rules Configured:
root > auditctl -l
LIST_RULES: exit,always arch=3221225534 (0xc000003e) syscall=all
Attached is the program used to load the system.
Results:
Without enabling audit 12.29
With auditing enabled and no rules configured 12.31
With auditing enabled, 1 rule configured but auditd not running - kauditd logs audit records to syslog via printk 41.02
The degradation is around 200%
Regards,
Logeswari.
-----Original Message-----
From: Richard Guy Briggs [mailto:rgb@redhat.com]
Sent: Wednesday, February 04, 2015 9:46 PM
To: Viswanath, Logeswari P (MCOU OSTL)
Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
Subject: Re: Linux audit performance impact
On 15/02/04, Viswanath, Logeswari P (MCOU OSTL) wrote:
> The intent is to calculate the performance impact by the auditing
> components such as
>
> 1) impact because of kauditd without auditd - but kauditd writes to syslog, so we are unable to determine the impact just because of kauditd - It is fine even if the audit record is dropped by kauditd. Is there any way to do this?
Not yet. That is a mode that has not been useful to anyone yet. You are welcome to hack a custom kernel to disable klog for doing testing instrumentation.
> 2) impact because of running auditd - log format NOLOG
> 3) impact because of running audispd - small plugin is written which will just read the audit records and doesn't processes it.
>
> -----Original Message-----
> From: Richard Guy Briggs [mailto:rgb@redhat.com]
> Sent: Tuesday, February 03, 2015 10:33 PM
> To: Satish Chandra Kilaru
> Cc: Viswanath, Logeswari P (MCOU OSTL); Steve Grubb;
> linux-audit@redhat.com
> Subject: Re: Linux audit performance impact
>
> On 15/02/03, Satish Chandra Kilaru wrote:
> > Thanks for The info. But my question was rhetorical... I meant to
> > say that it would not be much... She is trying to bombard the system
> > with open calls ... So lots and lots of events will be generated and
> > kernel has to write down the events some where or discard them...
>
> Exactly. It is of little practical use. You have to do I/O at some point, either to the same disk or another, or to a network interface or serial port, otherwise, just chuck it out. You could do a performance measurement on a short burst, then drain the queue, but what will that actually tell us?
>
> > On Tuesday, February 3, 2015, Richard Guy Briggs <rgb@redhat.com> wrote:
> >
> > > On 15/02/03, Satish Chandra Kilaru wrote:
> > > > How many events can kernel accumulate without I/o ?
> > >
> > > The kernel default is 64 *buffers*, but I think Fedora and RHEL
> > > set it to 320. It is now possible to set it to "0" which means
> > > limited only by system resources. See "man auditctl", "-b"
> > > option. An event can be made up of several buffers.
> > >
> > > Of course, how long a system lasts before the queue blows up
> > > depends on your rule set...
> > >
> > > However, at the moment, it will still write out to klog if auditd
> > > isn't running.
> > >
> > > > On Tuesday, February 3, 2015, Viswanath, Logeswari P (MCOU OSTL)
> > > > < logeswari.pv@hp.com <javascript:;>> wrote:
> > > >
> > > > > I don't want to disable auditing (i.e. disable audit record
> > > collection),
> > > > > but just do not want the records to delivered to user space
> > > > > since I
> > > want to
> > > > > remove the I/O overhead while running the performance test.
> > > > > Is there any option for this?
> > > > >
> > > > > -----Original Message-----
> > > > > From: Richard Guy Briggs [mailto:rgb@redhat.com <javascript:;>
> > > <javascript:;>]
> > > > > Sent: Thursday, January 29, 2015 10:23 PM
> > > > > To: Viswanath, Logeswari P (MCOU OSTL)
> > > > > Cc: Satish Chandra Kilaru; Steve Grubb; linux-audit@redhat.com
> > > <javascript:;>
> > > > > <javascript:;>
> > > > > Subject: Re: Linux audit performance impact
> > > > >
> > > > > On 15/01/29, Viswanath, Logeswari P (MCOU OSTL) wrote:
> > > > > > Please read my question as “Is there any option to configure
> > > > > > kaudit not to log audit records to syslog? when auditd not running.”
> > > > >
> > > > > Yeah, remove audit=1 from the kernel command line, or set
> > > > > audit=0 in
> > > its
> > > > > place. This will stop all but AVCs and if auditd has ever run
> > > > > since
> > > boot.
> > > > > If audit=0 is on the kernel boot line, it will be impossible
> > > > > to run
> > > auditd.
> > > > >
> > > > > There is a feature request that is likely coming soon that
> > > > > could be
> > > > > useful:
> > > > >
> > > > > https://bugzilla.redhat.com/show_bug.cgi?id=1160046
> > > > > "If no audit daemon is running, but an audit multicast
> > > > > subscriber is around, then the kernel shouldn't forward audit data to kmsg"
> > > > >
> > > > > > From: Viswanath, Logeswari P (MCOU OSTL)
> > > > > > Sent: Thursday, January 29, 2015 11:49 AM
> > > > > > To: 'Satish Chandra Kilaru'; Steve Grubb
> > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > Subject: RE: Linux audit performance impact
> > > > > >
> > > > > > Is there any option to configure kaudit not to log audit
> > > > > > records to
> > > > > syslog when auditd is running?
> > > > > > This way we can assess the impact of enabling audit without
> > > > > > involving
> > > > > disk I/o overhead.
> > > > > >
> > > > > > From: Satish Chandra Kilaru [mailto:iam.kilaru@gmail.com
> > > <javascript:;> <javascript:;>]
> > > > > > Sent: Thursday, January 29, 2015 9:12 AM
> > > > > > To: Steve Grubb
> > > > > > Cc: linux-audit@redhat.com <javascript:;> <javascript:;><mailto:
> > > linux-audit@redhat.com <javascript:;>
> > > > > <javascript:;>>; Viswanath,
> > > > > > Logeswari P (MCOU OSTL)
> > > > > > Subject: Re: Linux audit performance impact
> > > > > >
> > > > > > I agree with you... but writing to disk can trigger further
> > > > > > events
> > > > > leading spiralling of events...
> > > > > > I brought down my server few times with stupid rules...
> > > > > >
> > > > > > On Wed, Jan 28, 2015 at 10:39 PM, Steve Grubb
> > > > > > <sgrubb@redhat.com
> > > <javascript:;>
> > > > > <javascript:;><mailto:sgrubb@redhat.com <javascript:;>
> > > <javascript:;>>> wrote:
> > > > > > On Wednesday, January 28, 2015 10:18:47 AM Satish Chandra
> > > > > > Kilaru
> > > wrote:
> > > > > > > Write your own program to receive audit events directly
> > > > > > > without using auditd...
> > > > > > > That should be faster ....
> > > > > > > Auditd will log the events to disk causing more I/o than u need...
> > > > > >
> > > > > > But even that is configurable in many ways. You can decide
> > > > > > if you
> > > want
> > > > > > logging to disk or not and what kind of assurance that it
> > > > > > made it to disk and the priority of that audit daemon. Then
> > > > > > you also have all
> > > the
> > > > > > normal tuning knobs for disk throughput that you would use
> > > > > > for any disk performance critical system.
> > > > > >
> > > > > > -Steve
> > > > > >
> > > > > > > On Wednesday, January 28, 2015, Viswanath, Logeswari P
> > > > > > > (MCOU
> > > > > > > OSTL)
> > > <
> > > > > > >
> > > > > > > logeswari.pv@hp.com <javascript:;> <javascript:;><mailto:
> > > logeswari.pv@hp.com <javascript:;>
> > > > > <javascript:;>>> wrote:
> > > > > > > > Hi Steve,
> > > > > > > >
> > > > > > > > I am Logeswari working for HP.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > We want to know audit performance impact on RHEL and
> > > > > > > > Suse linux
> > > to
> > > > > > > > help us evaluate linux audit as data source for our host
> > > > > > > > based
> > > IDS.
> > > > > > > >
> > > > > > > > When we ran our own performance test with a test audispd
> > > > > > > > plugin, we found if a system can perform 200000
> > > > > > > > open/close system calls per second without auditing,
> > > > > > > > system can perform only 3000 open/close system calls
> > > > > > > > auditing is enabled for open/close system call which is
> > > > > > > > a HUGE impact on the system performance. It would
> > > be
> > > > > > > > great if anyone can help us answering the following questions.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > 1) Is this performance impact expected? If yes, what is the
> > > > > reason
> > > > > > > > behind it and can we fix it?
> > > > > > > >
> > > > > > > > 2) Have anyone done any benchmarking for performance
> > > impact? If
> > > > > yes,
> > > > > > > > can you please share the numbers and also the
> > > > > > > > steps/programs used the run the same.
> > > > > > > >
> > > > > > > > 3) Help us validating the performance test we have done in
> > > our
> > > > > test
> > > > > > > > setup using the steps mentioned along with the results attached.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Attached test program (loader.c) to invoke open and
> > > > > > > > close system
> > > > > calls.
> > > > > > > >
> > > > > > > > Attached idskerndsp is the audispd plugin program.
> > > > > > > >
> > > > > > > > We used time command to determine how much time the
> > > > > > > > system took
> > > to
> > > > > > > > complete 50000 open/close system calls without (results
> > > > > > > > attached
> > > > > > > > Without-auditing) and with auditing enabled on the
> > > > > > > > system (With-auditing-NOLOG-audispd-plugin and
> > > > > > > > With-auditing-RAW)
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > System details:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > 1 CPU machine
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *OS Version*
> > > > > > > >
> > > > > > > > RHEL 6.5
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *Kernel Version*
> > > > > > > >
> > > > > > > > uname –r
> > > > > > > >
> > > > > > > > 2.6.32-431.el6.x86_64
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Note: auditd was occupying 35% of CPU and was sleeping
> > > > > > > > for most
> > > of
> > > > > > > > the time whereas kauditd was occupying 20% of the CPU.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks & Regards,
> > > > > > > >
> > > > > > > > Logeswari.
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Please Donate to www.wikipedia.org<http://www.wikipedia.org>
> > > > >
> > > > > > --
> > > > > > Linux-audit mailing list
> > > > > > Linux-audit@redhat.com <javascript:;> <javascript:;>
> > > > > > https://www.redhat.com/mailman/listinfo/linux-audit
> > > > >
> > > > >
> > > > > - RGB
> > > > >
> > > > > --
> > > > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>
> > > > > <javascript:;>> Senior Software Engineer, Kernel Security,
> > > > > AMER ENG Base Operating Systems, Red Hat Remote, Ottawa,
> > > > > Canada
> > > > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > > > +1.613.693.0684x3545
> > > > >
> > > >
> > > >
> > > > --
> > > > Please Donate to www.wikipedia.org
> > >
> > > - RGB
> > >
> > > --
> > > Richard Guy Briggs <rbriggs@redhat.com <javascript:;>> Senior
> > > Software Engineer, Kernel Security, AMER ENG Base Operating
> > > Systems, Red Hat Remote, Ottawa, Canada
> > > Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> > > +1.613.693.0684x3545
> > >
> >
> >
> > --
> > Please Donate to www.wikipedia.org
>
> - RGB
>
> --
> Richard Guy Briggs <rbriggs@redhat.com> Senior Software Engineer,
> Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote,
> Ottawa, Canada
> Voice: +1.647.777.2635, Internal: (81) 32635, Alt:
> +1.613.693.0684x3545
- RGB
--
Richard Guy Briggs <rbriggs@redhat.com>
Senior Software Engineer, Kernel Security, AMER ENG Base Operating Systems, Red Hat Remote, Ottawa, Canada
Voice: +1.647.777.2635, Internal: (81) 32635, Alt: +1.613.693.0684x3545
--
Linux-audit mailing list
Linux-audit@redhat.com
https://www.redhat.com/mailman/listinfo/linux-audit
^ permalink raw reply
* Re: [PATCH] fsnotify: Fix handling of renames in audit
From: Paul Moore @ 2015-02-11 13:28 UTC (permalink / raw)
To: Jan Kara; +Cc: Andrew Morton, LKML, linux-audit, Eric Paris, stable
In-Reply-To: <20150211063658.GA30939@quack.suse.cz>
On Wed, Feb 11, 2015 at 1:36 AM, Jan Kara <jack@suse.cz> wrote:
> On Tue 10-02-15 19:58:35, Paul Moore wrote:
>> I can confirm this is indeed broken.
>>
>> Thanks for the patch. I'm building a test kernel as I type this, assuming all
>> goes well I'll send this to Linus with the other audit patches for v3.20.
>
> Actually, Andrew has picked up the patch and already sent it to Linus.
> So you don't have to put it in your tree. I sent it to Andrew since he
> usually takes care of pushing fsnotify fixes to Linus and although this was
> an audit bug, the problem was really in fsnotify...
Okay, great. As long as it gets to Linus eventually I'm happy.
FWIW, your patch tested just fine on my system.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: [PATCH] fsnotify: Fix handling of renames in audit
From: Jan Kara @ 2015-02-11 6:36 UTC (permalink / raw)
To: Paul Moore; +Cc: Jan Kara, Andrew Morton, LKML, linux-audit, Eric Paris, stable
In-Reply-To: <20343600.HhDMLCt7mX@sifl>
On Tue 10-02-15 19:58:35, Paul Moore wrote:
> On Tuesday, February 10, 2015 04:00:12 PM Jan Kara wrote:
> > Commit e9fd702a58c4 (audit: convert audit watches to use fsnotify
> > instead of inotify) broke handling of renames in audit. Audit code wants
> > to update inode number of an inode corresponding to watched name in a
> > directory. When something gets renamed into a directory to a watched
> > name, inotify previously passed moved inode to audit code however new
> > fsnotify code passes directory inode where the change happened. That
> > confuses audit and it starts watching parent directory instead of a file
> > in a directory.
> >
> > This can be observed for example by doing:
> > cd /tmp
> > touch foo bar
> > auditctl -w /tmp/foo
> > touch foo
> > mv bar foo
> > touch foo
> >
> > In audit log we see events like:
> > type=CONFIG_CHANGE msg=audit(1423563584.155:90): auid=1000 ses=2
> > op="updated rules" path="/tmp/foo" key=(null) list=4 res=1
> > ...
> > type=PATH msg=audit(1423563584.155:91): item=2 name="bar" inode=1046884
> > dev=08:0
> > 2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=DELETE
> > type=PATH msg=audit(1423563584.155:91): item=3 name="foo" inode=1046842
> > dev=08:0
> > 2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=DELETE
> > type=PATH msg=audit(1423563584.155:91): item=4 name="foo" inode=1046884
> > dev=08:0
> > 2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=CREATE
> > ...
> > and that's it - we see event for the first touch after creating the
> > audit rule, we see events for rename but we don't see any event for the
> > last touch. However we start seeing events for unrelated stuff happening
> > in /tmp.
> >
> > Fix the problem by passing moved inode as data in the FS_MOVED_FROM and
> > FS_MOVED_TO events instead of the directory where the change happens.
> > This doesn't introduce any new problems because noone besides
> > audit_watch.c cares about the passed value:
> > fs/notify/fanotify/fanotify.c cares only about FSNOTIFY_EVENT_PATH events.
> > fs/notify/dnotify/dnotify.c doesn't care about passed 'data' value at all.
> > fs/notify/inotify/inotify_fsnotify.c uses 'data' only for
> > FSNOTIFY_EVENT_PATH. kernel/audit_tree.c doesn't care about passed 'data'
> > at all.
> > kernel/audit_watch.c expects moved inode as 'data'.
> >
> > CC: linux-audit@redhat.com
> > CC: Paul Moore <paul@paul-moore.com>
> > CC: Eric Paris <eparis@redhat.com>
> > CC: stable@vger.kernel.org
> > Fixes: e9fd702a58c49dbb14481dca88dad44758da393a
> > Signed-off-by: Jan Kara <jack@suse.cz>
> > ---
> > include/linux/fsnotify.h | 6 ++++--
> > 1 file changed, 4 insertions(+), 2 deletions(-)
>
> I can confirm this is indeed broken.
>
> Thanks for the patch. I'm building a test kernel as I type this, assuming all
> goes well I'll send this to Linus with the other audit patches for v3.20.
Actually, Andrew has picked up the patch and already sent it to Linus.
So you don't have to put it in your tree. I sent it to Andrew since he
usually takes care of pushing fsnotify fixes to Linus and although this was
an audit bug, the problem was really in fsnotify...
Honza
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
^ permalink raw reply
* Re: [PATCH] fsnotify: Fix handling of renames in audit
From: Paul Moore @ 2015-02-11 0:58 UTC (permalink / raw)
To: Jan Kara; +Cc: Andrew Morton, LKML, linux-audit, Eric Paris, stable
In-Reply-To: <1423580412-19958-1-git-send-email-jack@suse.cz>
On Tuesday, February 10, 2015 04:00:12 PM Jan Kara wrote:
> Commit e9fd702a58c4 (audit: convert audit watches to use fsnotify
> instead of inotify) broke handling of renames in audit. Audit code wants
> to update inode number of an inode corresponding to watched name in a
> directory. When something gets renamed into a directory to a watched
> name, inotify previously passed moved inode to audit code however new
> fsnotify code passes directory inode where the change happened. That
> confuses audit and it starts watching parent directory instead of a file
> in a directory.
>
> This can be observed for example by doing:
> cd /tmp
> touch foo bar
> auditctl -w /tmp/foo
> touch foo
> mv bar foo
> touch foo
>
> In audit log we see events like:
> type=CONFIG_CHANGE msg=audit(1423563584.155:90): auid=1000 ses=2
> op="updated rules" path="/tmp/foo" key=(null) list=4 res=1
> ...
> type=PATH msg=audit(1423563584.155:91): item=2 name="bar" inode=1046884
> dev=08:0
> 2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=DELETE
> type=PATH msg=audit(1423563584.155:91): item=3 name="foo" inode=1046842
> dev=08:0
> 2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=DELETE
> type=PATH msg=audit(1423563584.155:91): item=4 name="foo" inode=1046884
> dev=08:0
> 2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=CREATE
> ...
> and that's it - we see event for the first touch after creating the
> audit rule, we see events for rename but we don't see any event for the
> last touch. However we start seeing events for unrelated stuff happening
> in /tmp.
>
> Fix the problem by passing moved inode as data in the FS_MOVED_FROM and
> FS_MOVED_TO events instead of the directory where the change happens.
> This doesn't introduce any new problems because noone besides
> audit_watch.c cares about the passed value:
> fs/notify/fanotify/fanotify.c cares only about FSNOTIFY_EVENT_PATH events.
> fs/notify/dnotify/dnotify.c doesn't care about passed 'data' value at all.
> fs/notify/inotify/inotify_fsnotify.c uses 'data' only for
> FSNOTIFY_EVENT_PATH. kernel/audit_tree.c doesn't care about passed 'data'
> at all.
> kernel/audit_watch.c expects moved inode as 'data'.
>
> CC: linux-audit@redhat.com
> CC: Paul Moore <paul@paul-moore.com>
> CC: Eric Paris <eparis@redhat.com>
> CC: stable@vger.kernel.org
> Fixes: e9fd702a58c49dbb14481dca88dad44758da393a
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
> include/linux/fsnotify.h | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
I can confirm this is indeed broken.
Thanks for the patch. I'm building a test kernel as I type this, assuming all
goes well I'll send this to Linus with the other audit patches for v3.20.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: Audit: audisp-remote plugin: why is OpenLDAP a dependency ??
From: salgak @ 2015-02-10 19:01 UTC (permalink / raw)
To: Steve Grubb; +Cc: linux-audit
In-Reply-To: <1286467080.2438459.1423502775408.JavaMail.zimbra@comcast.net>
[-- Attachment #1.1: Type: text/plain, Size: 2985 bytes --]
So, respin a new RPM using rpmrebuild, dropping the zos plugins ?
i.e. the appropriate section of the SPEC would now look like this:
Provides: audit-audispd-plugins = 1.8-0.28.1
#Requires: openldap2
#Requires: rpmlib(PayloadFilesHavePrefix) <= 4.0-1
#Requires: rpmlib(CompressedFileNames) <= 3.0.4-1
Requires: libaudit.so.0()(64bit)
Requires: libauparse.so.0()(64bit)
Requires: libc.so.6()(64bit)
Requires: libc.so.6(GLIBC_2.2.5)(64bit)
Requires: libc.so.6(GLIBC_2.3)(64bit)
Requires: libc.so.6(GLIBC_2.3.4)(64bit)
Requires: libc.so.6(GLIBC_2.4)(64bit)
Requires: libc.so.6(GLIBC_2.8)(64bit)
#Requires: liblber-2.4.so.2()(64bit)
#Requires: libldap-2.4.so.2()(64bit)
Requires: libpthread.so.0()(64bit)
Requires: libpthread.so.0(GLIBC_2.2.5)(64bit)
Requires: libpthread.so.0(GLIBC_2.3.2)(64bit)
#Requires: rpmlib(PayloadIsLzma) <= 4.4.6-1
#suggest
#enhance
%description
The audit-audispd-plugins package contains plugin components for the
audit dispatcher (audispd).
Authors:
--------
Steve Grubb <sgrubb@redhat.com>
%files
%config(noreplace) %attr(0640, root, root) "/etc/audisp/audisp-remote.conf"
%config(noreplace) %attr(0640, root, root) "/etc/audisp/plugins.d/au-remote.conf"
#%config(noreplace) %attr(0640, root, root) "/etc/audisp/plugins.d/audispd-zos-remote.conf"
#%config(noreplace) %attr(0640, root, root) "/etc/audisp/zos-remote.conf"
%attr(0750, root, root) "/usr/sbin/audisp-remote"
#%attr(0750, root, root) "/usr/sbin/audispd-zos-remote"
%doc %attr(0644, root, root) "/usr/share/man/man5/audisp-remote.conf.5.gz"
#%doc %attr(0644, root, root) "/usr/share/man/man5/zos-remote.conf.5.gz"
%doc %attr(0644, root, root) "/usr/share/man/man8/audisp-remote.8.gz"
#%doc %attr(0644, root, root) "/usr/share/man/man8/audispd-zos-remote.8.gz"
----- Original Message -----
From: salgak@comcast.net
To: "Steve Grubb" <sgrubb@redhat.com>
Cc: linux-audit@redhat.com
Sent: Monday, February 9, 2015 12:26:15 PM
Subject: Re: Audit: audisp-remote plugin: why is OpenLDAP a dependency ??
Yeah, just crawled into the rpm and saw that. Going to try to re-spin a smaller RPM without the zos and see if that works.
----- Original Message -----
From: "Steve Grubb" <sgrubb@redhat.com>
To: salgak@comcast.net
Cc: linux-audit@redhat.com
Sent: Monday, February 9, 2015 12:00:58 PM
Subject: Re: Audit: audisp-remote plugin: why is OpenLDAP a dependency ??
On Mon, 9 Feb 2015 13:47:36 +0000 (UTC)
salgak@comcast.net wrote:
> I'm currently dealing with a VMware appliance, and need to export my
> audit logs. Despite VMware telling me to use the audisp-remote plugin
> with auditd, I get a dependency check for OpenLDAP, and the install
> fails.
>
> I'm pointing to my audit server (a Splunk box) via IP address. Is
> there any workaround, or am I stuck adding even MORE software to the
> appliance ??
The package the plugin is included in contains all of them. The
zos-remote plugin is the one that wants openldap.
-Steve
[-- Attachment #1.2: Type: text/html, Size: 5420 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply
* [PATCH] fsnotify: Fix handling of renames in audit
From: Jan Kara @ 2015-02-10 15:00 UTC (permalink / raw)
To: Andrew Morton; +Cc: LKML, Jan Kara, linux-audit, Paul Moore, Eric Paris, stable
Commit e9fd702a58c4 (audit: convert audit watches to use fsnotify
instead of inotify) broke handling of renames in audit. Audit code wants
to update inode number of an inode corresponding to watched name in a
directory. When something gets renamed into a directory to a watched
name, inotify previously passed moved inode to audit code however new
fsnotify code passes directory inode where the change happened. That
confuses audit and it starts watching parent directory instead of a file
in a directory.
This can be observed for example by doing:
cd /tmp
touch foo bar
auditctl -w /tmp/foo
touch foo
mv bar foo
touch foo
In audit log we see events like:
type=CONFIG_CHANGE msg=audit(1423563584.155:90): auid=1000 ses=2
op="updated rules" path="/tmp/foo" key=(null) list=4 res=1
...
type=PATH msg=audit(1423563584.155:91): item=2 name="bar" inode=1046884
dev=08:0
2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=DELETE
type=PATH msg=audit(1423563584.155:91): item=3 name="foo" inode=1046842
dev=08:0
2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=DELETE
type=PATH msg=audit(1423563584.155:91): item=4 name="foo" inode=1046884
dev=08:0
2 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=CREATE
...
and that's it - we see event for the first touch after creating the
audit rule, we see events for rename but we don't see any event for the
last touch. However we start seeing events for unrelated stuff happening
in /tmp.
Fix the problem by passing moved inode as data in the FS_MOVED_FROM and
FS_MOVED_TO events instead of the directory where the change happens.
This doesn't introduce any new problems because noone besides
audit_watch.c cares about the passed value:
fs/notify/fanotify/fanotify.c cares only about FSNOTIFY_EVENT_PATH events.
fs/notify/dnotify/dnotify.c doesn't care about passed 'data' value at all.
fs/notify/inotify/inotify_fsnotify.c uses 'data' only for FSNOTIFY_EVENT_PATH.
kernel/audit_tree.c doesn't care about passed 'data' at all.
kernel/audit_watch.c expects moved inode as 'data'.
CC: linux-audit@redhat.com
CC: Paul Moore <paul@paul-moore.com>
CC: Eric Paris <eparis@redhat.com>
CC: stable@vger.kernel.org
Fixes: e9fd702a58c49dbb14481dca88dad44758da393a
Signed-off-by: Jan Kara <jack@suse.cz>
---
include/linux/fsnotify.h | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/include/linux/fsnotify.h b/include/linux/fsnotify.h
index 1c804b057fb1..7ee1774edee5 100644
--- a/include/linux/fsnotify.h
+++ b/include/linux/fsnotify.h
@@ -101,8 +101,10 @@ static inline void fsnotify_move(struct inode *old_dir, struct inode *new_dir,
new_dir_mask |= FS_ISDIR;
}
- fsnotify(old_dir, old_dir_mask, old_dir, FSNOTIFY_EVENT_INODE, old_name, fs_cookie);
- fsnotify(new_dir, new_dir_mask, new_dir, FSNOTIFY_EVENT_INODE, new_name, fs_cookie);
+ fsnotify(old_dir, old_dir_mask, source, FSNOTIFY_EVENT_INODE, old_name,
+ fs_cookie);
+ fsnotify(new_dir, new_dir_mask, source, FSNOTIFY_EVENT_INODE, new_name,
+ fs_cookie);
if (target)
fsnotify_link_count(target);
--
2.1.4
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox