clone flags

clone flags

[John D. Ramsdell]

I've been carefully comparing output I obtain with autrace with what I
get from strace.  It appears they differ when the clone system call is
invoked from the C library via fork.  In particular, strace reports
flags of CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, while
autrace says the flags are 0.  The flags are in field a2.

John

[ramsdell@goo fork]$ uname -r
2.6.21-1.3228.fc7
[ramsdell@goo fork]$ make fork
cc     fork.c   -o fork
[ramsdell@goo fork]$ strace -o strace.txt ./fork
[ramsdell@goo fork]$ su -
Password: 
[root@goo ~]# cd /home/ramsdell/proj/fork
[root@goo fork]# autrace ./fork
Waiting to execute: ./fork
Cleaning up...
Trace complete. You can locate the records with 'ausearch -i -p 1160'
[root@goo fork]# ausearch -i -p 1160 > autrace.txt
[root@goo fork]# grep clone strace.txt 
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb7efb708) = 1122
[root@goo fork]# grep clone autrace.txt 
type=SYSCALL msg=audit(07/19/2007 09:16:02.350:848) : arch=i386 syscall=clone success=yes exit=1161 a0=1200011 a1=0 a2=0 a3=0 items=0 ppid=1158 pid=1160 auid=ramsdell uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts2 comm=fork exe=/home/ramsdell/proj/fork/fork subj=user_u:system_r:unconfined_t:s0 key=(null) 
[root@goo fork]# cat fork.c 
#include <stdio.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/wait.h>

int main(int argc, char **argv)
{
  int status;
  pid_t pid;
  switch (fork()) {
  case -1:
    perror("clone");
    return 1;
  case 0:
    return 0;
  default:
    do {
      pid = wait(&status);
    } while (pid < 0 && errno == EINTR);
    if (WIFEXITED(status))
      return WEXITSTATUS(status);
    else
      return 1;
  }
}
[root@goo fork]# 

Re: clone flags

[Eric Paris]

On Thu, 2007-07-19 at 09:24 -0400, John D. Ramsdell wrote:

> [root@goo fork]# ausearch -i -p 1160 > autrace.txt
> [root@goo fork]# grep clone strace.txt 
> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb7efb708) = 1122
> [root@goo fork]# grep clone autrace.txt 
> type=SYSCALL msg=audit(07/19/2007 09:16:02.350:848) : arch=i386 syscall=clone success=yes exit=1161 a0=1200011 a1=0 a2=0 a3=0 items=0 ppid=1158 pid=1160 auid=ramsdell uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=pts2 comm=fork exe=/home/ramsdell/proj/fork/fork subj=user_u:system_r:unconfined_t:s0 key=(null) 

Actually it's a problem with mapping things.  The flags are in a0.  If
you look at the clone man page they talk about sys_clone at the bottom
(which is the actual call, whereas cone is just a library function on
top of the call) and they state the the ordering for sys_clone is
different.  The kernel function is actually

asmlinkage long
sys_clone(unsigned long clone_flags, unsigned long newsp,
          void __user *parent_tid, void __user *child_tid, struct
pt_regs *regs)

So the flags are actually coming in the first argument.  To verify check

#define CLONE_CHILD_SETTID   0x01000000
#define CLONE_CHILD_CLEARTID 0x00200000
#define SIGCHLD              0x00000011

Which just so happens to be  0x01200011
and a0 just so happen to be     1200011

But it's just a difference between the library call 'clone' that the
application makes and the actual syscall glibc translates that to
sys_clone and the ordering of the flags.

-Eric

Re: clone flags

[John D. Ramsdell]

Eric Paris <eparis@redhat.com> writes:

> Actually it's a problem with mapping things.  The flags are in a0.

Eric, you seem to have nailed this issue.

Let me explain why I thought the flags are in a2.  I read print_a2
which starts on line 1019 of auparse/interpret.c.  It contains a call
to print_clone_flags on line 1034, and there are no other calls to
this function in that file.  That code clearly assumes clone flags
only occur in a2.

I played around with tracing the clone system call, and found
something else since my last message.  I traced a program that creates
threads within a single address space with clone, and that call puts
the clone flags into a2.  The auparse library interprets these flags
as one would expect.

It is interesting to note that strace shows the clone flags for both
usages of clone in the same position.  Strace tries to make its output
more human readable by making system calls records look more uniform
then they actually are.

The lack of uniformity in the output for interpreted audit system call
will increase the difficulty of analyzing traces for people like me.
In this case, I'll have to figure out when clone flags are in a0 and
when they are in a2.  The auparse library will have to do the same.

My processing pipeline allows me to print out traces records as tab
separated values so that I can probe data with quick awk scripts.  Off
the top of my head, it's not clear to me how to distinguish two case.
I have enclosed the output with which I am working.  I suppose I
should simply read the strace sources, as its authors clearly have
already figured out how to resolve this issue.

John

autrace of pthread_create:

event time	1183463049.249:3627	type	SYSCALL	arch	i386	syscall	clone	success	yes	exit	14871	a0	3d0f00	a1	b7efb4b4	a2	CLONE_VM|CLONE_FS|CLONE_SIGHAND|CLONE_PTRACE|CLONE_PARENT|CLONE_THREAD|CLONE_NEWNS|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_CHILD_CLEARTID|CLONE_DETACHED|CLONE_UNTRACED|CLONE_CHILD_SETTID|CLONE_STOPPED|CLONE_NEWUTS	a3	bf9823ac	items	0	ppid	14857	pid	14858	auid	ramsdell	uid	ramsdell	gid	ramsdell	euid	ramsdell	suid	ramsdell	fsuid	ramsdell	egid	ramsdell	sgid	ramsdell	fsgid	ramsdell	tty	pts0	comm	sockpair	exe	/home/ramsdell/scm/polgen/src/daemon-example/sockpair	subj	system_u:system_r:unconfined_t:s0	key	(null)

autrace of fork:

event time	1183463044.249:414	type	SYSCALL	arch	i386	syscall	clone	success	yes	exit	14860	a0	1200011	a1	0	a2	0	a3	0	items	0	ppid	14858	pid	14859	auid	ramsdell	uid	ramsdell	gid	ramsdell	euid	ramsdell	suid	ramsdell	fsuid	ramsdell	egid	ramsdell	sgid	ramsdell	fsgid	ramsdell	tty	pts0	comm	broadcast	exe	/home/ramsdell/scm/polgen/src/daemon-example/broadcast	subj	system_u:system_r:unconfined_t:s0	key	(null)


strace of ptrace_create:

31999	clone	child_stack=0xb7f7f4b4	flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID	parent_tidptr=0xb7f7fbd8	{entry_number:6, base_addr:0xb7f7fb90, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}	child_tidptr=0xb7f7fbd8	32013	user_u:system_r:unconfined_t

strace of fork:

31999	clone	child_stack=0	flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD	child_tidptr=0xb7f9e708	32000	user_u:system_r:unconfined_t


John

Re: clone flags

[Eric Paris]

On Thu, 2007-07-19 at 14:45 -0400, John D. Ramsdell wrote: 
> Eric Paris <eparis@redhat.com> writes:
> 
> > Actually it's a problem with mapping things.  The flags are in a0.
> 
> Eric, you seem to have nailed this issue.

> I played around with tracing the clone system call, and found
> something else since my last message.  I traced a program that creates
> threads within a single address space with clone, and that call puts
> the clone flags into a2.  The auparse library interprets these flags
> as one would expect.

Actually, not quite.  Its still the same mapping problem.  auparse is
busted and the flags are always in a0 for the audit log.  auparse is
actually giving you total and complete crap output.  Notice in auparse
you got a long list of flags.

CLONE_VM|CLONE_FS|CLONE_SIGHAND|CLONE_PTRACE|CLONE_PARENT|CLONE_THREAD|
CLONE_NEWNS|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_CHILD_CLEARTID|
CLONE_DETACHED|CLONE_UNTRACED|CLONE_CHILD_SETTID|CLONE_STOPPED|
CLONE_NEWUTS

And the strace output below only showed a short list of flags.  Also
note that one is NOT a sub/super set of the other.  CLONE_DETACHED above
and CLONE_FILES below?

CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|
CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID

Anyway I took the strace flags and worked it out:

#define CLONE_VM                 0x00000100
#define CLONE_FS                 0x00000200
#define CLONE_FILES              0x00000400
#define CLONE_SIGHAND            0x00000800
#define CLONE_THREAD             0x00010000
#define CLONE_SYSVSEM            0x00040000
#define CLONE_SETTLS             0x00080000
#define CLONE_PARENT_SETTID      0x00100000
#define CLONE_CHILD_CLEARTID     0x00200000

                                 0x003D0F00

Low and behold the audit a0 is       3d0f00

Looks like auparse was wrongly trying to convert the pointer for
parent_tidptr=0xb7f7fbd8 (notice we had CLONE_PARENT_SETTID set) into
clone flags and that list of flags was the best it could do.  

So I'd say change all your stuff to look only at a0 for clone and
someone (sgrubb already knows) needs to fix auparse to look for the
flags in a0 not in a2.

-Eric

Re: clone flags

[John D. Ramsdell]

Eric Paris <eparis@redhat.com> writes:

> So I'd say change all your stuff to look only at a0 for clone and
> someone (sgrubb already knows) needs to fix auparse to look for the
> flags in a0 not in a2.

This is very good news.  I guess I was too trusting of the correctness
of Steve's code.  Thank you.

John

Re: clone flags

[John D. Ramsdell]

Eric Paris <eparis@redhat.com> writes:

> So I'd say change all your stuff to look only at a0 for clone and
> someone (sgrubb already knows) needs to fix auparse to look for the
> flags in a0 not in a2.

I notice the name of the getdents64 system call is printed as
getdents.  I'll carefully study the output of strace and autrace on
all the system calls I monitor, and supply a patch that fixes
discrepancies.  The code in auparse/interpret.c seems straightforward.

John

Clone and fcntl64 flags patch

[John D. Ramsdell]

[-- Attachment #1: Type: text/plain, Size: 642 bytes --]

Enclosed is a patch for auparse/interpret.c that makes it so that
a0 is interpreted for clone flags, not a2.  It also fixes two problems
with interpreting the fcntl system call.  The name of the system call
is fcntl64, but the original code looked for the name fcntl.  I have
also added a case so that a2 is printed as FD_CLOEXEC whenever a1 is
F_SETFD and a2 is 1.

I still haven't figured out why the auparse library prints getdents
when strace print getdents64.  I'll keep on looking.  You'd think that
either both getdents and fcntl would be printed with or without the
64 tacked on, but the current situation seem very odd to me.

John


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: clone fcntl64 --]
[-- Type: text/x-patch, Size: 7789 bytes --]

Only in audit-1.5.5/audisp: audispd
Only in audit-1.5.5/audisp: audispd-audispd.o
Only in audit-1.5.5/audisp: .libs
Only in audit-1.5.5/audisp: Makefile
Only in audit-1.5.5/auparse: auditd-config.lo
Only in audit-1.5.5/auparse: auditd-config.o
Only in audit-1.5.5/auparse: auparse.lo
Only in audit-1.5.5/auparse: auparse.o
Only in audit-1.5.5/auparse: autsv
Only in audit-1.5.5/auparse: autsv__00.txt
Only in audit-1.5.5/auparse: autsv.c
Only in audit-1.5.5/auparse: autsv.txt
Only in audit-1.5.5/auparse: data_buf.lo
Only in audit-1.5.5/auparse: data_buf.o
Only in audit-1.5.5/auparse: .deps
Only in audit-1.5.5/auparse: ellist.lo
Only in audit-1.5.5/auparse: ellist.o
diff -ur oaudit-1.5.5/auparse/interpret.c audit-1.5.5/auparse/interpret.c
--- oaudit-1.5.5/auparse/interpret.c	2007-07-20 08:19:18.000000000 -0400
+++ audit-1.5.5/auparse/interpret.c	2007-07-23 07:30:42.000000000 -0400
@@ -978,9 +978,20 @@
 static const char *print_a0(const char *val, const rnode *r)
 {
 	int machine = r->machine, syscall = r->syscall;
+	char *out;
 	const char *sys = audit_syscall_to_name(syscall, machine);
 	if (sys) {
-		/* Unused right now... */
+		if (strcmp(sys, "clone") == 0) {
+			int ival;
+
+			errno = 0;
+			ival = strtoul(val, NULL, 16);
+		        if (errno) {
+                		asprintf(&out, "conversion error(%s)", val);
+	                	return out;
+	        	}
+			return print_clone_flags(ival);
+		}
 	}
 	return strdup(val);
 }
@@ -1001,7 +1012,7 @@
 	                	return out;
 	        	}
 			return print_open_flags(ival);
-		} else if (strncmp(sys, "fcntl", 5) == 0) {
+		} else if (strcmp(sys, "fcntl64") == 0) {
 			int ival;
 
 			errno = 0;
@@ -1022,17 +1033,7 @@
 	char *out;
 	const char *sys = audit_syscall_to_name(syscall, machine);
 	if (sys) {
-		if (strcmp(sys, "clone") == 0) {
-			int ival;
-
-			errno = 0;
-			ival = strtoul(val, NULL, 16);
-		        if (errno) {
-                		asprintf(&out, "conversion error(%s)", val);
-	                	return out;
-	        	}
-			return print_clone_flags(ival);
-		} else if (strncmp(sys, "fcntl", 5) == 0) {
+		if (strcmp(sys, "fcntl64") == 0) {
 			int ival;
 
 			errno = 0;
@@ -1045,6 +1046,10 @@
 			{
 				case F_SETOWN:
 					return print_uid(val);
+				case F_SETFD:
+					if (ival == FD_CLOEXEC)
+						return strdup("FD_CLOEXEC");
+					/* Fall thru okay. */
 				case F_SETFL:
 				case F_SETLEASE:
 				case F_GETLEASE:
Only in audit-1.5.5/auparse: interpret.c~
Only in audit-1.5.5/auparse: interpret.lo
Only in audit-1.5.5/auparse: interpret.o
Only in audit-1.5.5/auparse: libauparse.la
Only in audit-1.5.5/auparse: .libs
Only in audit-1.5.5/auparse: Makefile
Only in audit-1.5.5/auparse: message.lo
Only in audit-1.5.5/auparse: message.o
Only in audit-1.5.5/auparse: nvlist.lo
Only in audit-1.5.5/auparse: nvlist.o
Only in audit-1.5.5/auparse: nvpair.lo
Only in audit-1.5.5/auparse: nvpair.o
Only in audit-1.5.5/auparse: oautsv
Only in audit-1.5.5/auparse: oautsv.txt
Only in audit-1.5.5/auparse: rlist.lo
Only in audit-1.5.5/auparse: rlist.o
Only in audit-1.5.5/auparse/test: .deps
Only in audit-1.5.5/auparse/test: Makefile
Only in audit-1.5.5/bindings: Makefile
Only in audit-1.5.5/bindings/python: build
Only in audit-1.5.5/bindings/python: Makefile
Only in audit-1.5.5: config.h
Only in audit-1.5.5: config.log
Only in audit-1.5.5: config.status
Only in audit-1.5.5/docs: Makefile
Only in audit-1.5.5/init.d: Makefile
Only in audit-1.5.5/lib: audit_logging.lo
Only in audit-1.5.5/lib: audit_logging.o
Only in audit-1.5.5/lib: deprecated.lo
Only in audit-1.5.5/lib: deprecated.o
Only in audit-1.5.5/lib: .deps
Only in audit-1.5.5/lib: libaudit.la
Only in audit-1.5.5/lib: libaudit.lo
Only in audit-1.5.5/lib: libaudit.o
Only in audit-1.5.5/lib: .libs
Only in audit-1.5.5/lib: lookup_table.lo
Only in audit-1.5.5/lib: lookup_table.o
Only in audit-1.5.5/lib: Makefile
Only in audit-1.5.5/lib: message.lo
Only in audit-1.5.5/lib: message.o
Only in audit-1.5.5/lib: netlink.lo
Only in audit-1.5.5/lib: netlink.o
Only in audit-1.5.5: libtool
Only in audit-1.5.5: Makefile
Only in audit-1.5.5/src: auditctl
Only in audit-1.5.5/src: auditctl-llist.o
Only in audit-1.5.5/src: auditctl.o
Only in audit-1.5.5/src: auditd
Only in audit-1.5.5/src: auditd-auditd-config.o
Only in audit-1.5.5/src: auditd-auditd-dispatch.o
Only in audit-1.5.5/src: auditd-auditd-event.o
Only in audit-1.5.5/src: auditd-auditd.o
Only in audit-1.5.5/src: auditd-auditd-reconfig.o
Only in audit-1.5.5/src: auditd-auditd-sendmail.o
Only in audit-1.5.5/src: auditd-config.o
Only in audit-1.5.5/src: aureport
Only in audit-1.5.5/src: aureport.o
Only in audit-1.5.5/src: aureport-options.o
Only in audit-1.5.5/src: aureport-output.o
Only in audit-1.5.5/src: aureport-scan.o
Only in audit-1.5.5/src: ausearch
Only in audit-1.5.5/src: ausearch-avc.o
Only in audit-1.5.5/src: ausearch-int.o
Only in audit-1.5.5/src: ausearch-llist.o
Only in audit-1.5.5/src: ausearch-lookup.o
Only in audit-1.5.5/src: ausearch-match.o
Only in audit-1.5.5/src: ausearch-nvpair.o
Only in audit-1.5.5/src: ausearch.o
Only in audit-1.5.5/src: ausearch-options.o
Only in audit-1.5.5/src: ausearch-parse.o
Only in audit-1.5.5/src: ausearch-report.o
Only in audit-1.5.5/src: ausearch-string.o
Only in audit-1.5.5/src: ausearch-time.o
Only in audit-1.5.5/src: autrace
Only in audit-1.5.5/src: autrace.o
Only in audit-1.5.5/src: delete_all.o
Only in audit-1.5.5/src: .libs
Only in audit-1.5.5/src: Makefile
Only in audit-1.5.5/src/mt: actiontab.h
Only in audit-1.5.5/src/mt: alpha_table.h
Only in audit-1.5.5/src/mt: audit_logging.o
Only in audit-1.5.5/src/mt: deprecated.o
Only in audit-1.5.5/src/mt: fieldtab.h
Only in audit-1.5.5/src/mt: flagtab.h
Only in audit-1.5.5/src/mt: i386_table.h
Only in audit-1.5.5/src/mt: ia64_table.h
Only in audit-1.5.5/src/mt: libauditmt.a
Only in audit-1.5.5/src/mt: libaudit.o
Only in audit-1.5.5/src/mt: lookup_table.o
Only in audit-1.5.5/src/mt: machinetab.h
Only in audit-1.5.5/src/mt: Makefile
Only in audit-1.5.5/src/mt: message.o
Only in audit-1.5.5/src/mt: msg_typetab.h
Only in audit-1.5.5/src/mt: netlink.o
Only in audit-1.5.5/src/mt: optab.h
Only in audit-1.5.5/src/mt: ppc_table.h
Only in audit-1.5.5/src/mt: s390_table.h
Only in audit-1.5.5/src/mt: s390x_table.h
Only in audit-1.5.5/src/mt: x86_64_table.h
Only in audit-1.5.5: stamp-h1
Only in audit-1.5.5/swig: _audit.la
Only in audit-1.5.5/swig: audit_wrap.c
Only in audit-1.5.5/swig: audit_wrap.lo
Only in audit-1.5.5/swig: audit_wrap.o
Only in audit-1.5.5/swig: .deps
Only in audit-1.5.5/swig: .libs
Only in audit-1.5.5/swig: Makefile
Only in audit-1.5.5/system-config-audit: config.log
Only in audit-1.5.5/system-config-audit: config.status
Only in audit-1.5.5/system-config-audit: intltool-extract
Only in audit-1.5.5/system-config-audit: intltool-merge
Only in audit-1.5.5/system-config-audit: intltool-update
Only in audit-1.5.5/system-config-audit: libtool
Only in audit-1.5.5/system-config-audit: Makefile
Only in audit-1.5.5/system-config-audit/po: .intltool-merge-cache
Only in audit-1.5.5/system-config-audit/po: Makefile
Only in audit-1.5.5/system-config-audit/po: Makefile.in
Only in audit-1.5.5/system-config-audit/po: POTFILES
Only in audit-1.5.5/system-config-audit/po: stamp-it
Only in audit-1.5.5/system-config-audit/src: config.h
Only in audit-1.5.5/system-config-audit/src: .deps
Only in audit-1.5.5/system-config-audit/src: .dirstamp
Only in audit-1.5.5/system-config-audit/src: .libs
Only in audit-1.5.5/system-config-audit/src: src_system_config_audit_server-server.o
Only in audit-1.5.5/system-config-audit/src: stamp-h1
Only in audit-1.5.5/system-config-audit/src: system-config-audit
Only in audit-1.5.5/system-config-audit/src: system-config-audit-server
Only in audit-1.5.5/system-config-audit: system-config-audit.desktop

[-- Attachment #3: Type: text/plain, Size: 0 bytes --]

Re: Clone and fcntl64 flags patch

[John D. Ramsdell]

ramsdell@mitre.org (John D. Ramsdell) writes:

> I still haven't figured out why the auparse library prints getdents
> when strace print getdents64.

There is no bug in auparse when it comes to printing getdents.  I
misread the strace output.  It is printing getdents, not getdents64,
just as is auparse.  My strace analysis programs looked for
getdents64, not getdents.  I guess there was a time when the 64 bit
version of the system call was in use, but it doesn't seem to be used
now.

John

Re: Clone and fcntl64 flags patch

[Steve Grubb]

On Monday 23 July 2007 07:44:42 am John D. Ramsdell wrote:
> Enclosed is a patch for auparse/interpret.c that makes it so that
> a0 is interpreted for clone flags, not a2.  

Thanks...will appy.

> It also fixes two problems with interpreting the fcntl system call.  The
> name of the system call is fcntl64, but the original code looked for the
> name fcntl.

It was doing: strncmp(sys, "fcntl", 5) == 0), which is not a full string 
compare. I think this is correct.


> I have  also added a case so that a2 is printed as FD_CLOEXEC whenever a1 is
> F_SETFD and a2 is 1.

Thanks...merging this piece.

-Steve

Re: Clone and fcntl64 flags patch

[John D. Ramsdell]

Steve Grubb <sgrubb@redhat.com> writes:

> It was doing: strncmp(sys, "fcntl", 5) == 0), which is not a full
> string compare. I think this is correct.

Yes.  You are right.  Sorry about that.

John