Re: [Feature Request?] Inline compression of process core dumps

[Andi Kleen <andi@firstfloor.org>]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > I saw that too, and unfortunately I don't know what what that condition 
> > represents, either.  It's the only other element in that if statement 
> > that could make it take that path, so I'm assuming that's part of the 
> > problem.
> 
> Multiple mm's mean multiple threads with a different set of mappings,
> which would fit for UML. Either way there should be a check for !pipe
> before appending the pid

Here's a patch. It just doesn't do any formatting for the pipe case.

-Andi

Fix core to pipe for multithreaded processes

I also removed the BKL around format_corename because it seems unneeded.

Signed-off-by: Andi Kleen <ak@suse.de>

Index: linux-2.6.21-rc3-test/fs/exec.c
===================================================================
--- linux-2.6.21-rc3-test.orig/fs/exec.c
+++ linux-2.6.21-rc3-test/fs/exec.c
@@ -1501,9 +1501,6 @@ int do_coredump(long signr, int exit_cod
 	 * lock_kernel() because format_corename() is controlled by sysctl, which
 	 * uses lock_kernel()
 	 */
- 	lock_kernel();
-	format_corename(corename, core_pattern, signr);
-	unlock_kernel();
  	if (corename[0] == '|') {
 		/* SIGPIPE can happen, but it's just never processed */
  		if(call_usermodehelper_pipe(corename+1, NULL, NULL, &file)) {
@@ -1512,10 +1509,12 @@ int do_coredump(long signr, int exit_cod
  			goto fail_unlock;
  		}
 		ispipe = 1;
- 	} else
+ 	} else {
+		format_corename(corename, core_pattern, signr);
  		file = filp_open(corename,
 				 O_CREAT | 2 | O_NOFOLLOW | O_LARGEFILE | flag,
 				 0600);
+	}
 	if (IS_ERR(file))
 		goto fail_unlock;
 	inode = file->f_path.dentry->d_inode;