linuxppc-dev.lists.ozlabs.org archive mirror
 help / color / mirror / Atom feed
* ppc64 sys_ipc breakage in 2.6.34-rc2
@ 2010-03-22  6:47 Anton Blanchard
  2010-03-22 18:00 ` Andreas Schwab
  2010-03-22 20:01 ` Christoph Hellwig
  0 siblings, 2 replies; 6+ messages in thread
From: Anton Blanchard @ 2010-03-22  6:47 UTC (permalink / raw)
  To: Christoph Hellwig, Ralf Baechle, Benjamin Herrenschmidt,
	Paul Mundt, Jeff Dike, Hirokazu Takata, Thomas Gleixner,
	Ingo Molnar, H. Peter Anvin, Al Viro, Arnd Bergmann,
	Heiko Carstens, Martin Schwidefsky, Luck, Tony, James Morris,
	Andreas Schwab, Jesper Nilsson, Russell King, David Howells,
	Kyle McMartin, Andrew Morton, Linus Torvalds
  Cc: linuxppc-dev


I chased down a fail on ppc64 on 2.6.34-rc2 where an application that uses
shared memory was getting a SEGV.

Commit baed7fc9b580bd3fb8252ff1d9b36eaf1f86b670 (Add generic sys_ipc wrapper)
changed the second argument from an unsigned long to an int. When we call
shmget the system call wrappers for sys_ipc will sign extend second (ie the
size) which truncates it. It took a while to track down because the call
succeeds and strace shows the untruncated size :)

The patch below changes second from an int to an unsigned long which fixes
shmget on ppc64 (and I assume s390, sparc64 and mips64).

Signed-off-by: Anton Blanchard <anton@samba.org>
--

I assume the function prototypes for the other IPC methods would cause us
to sign or zero extend second where appropriate (avoiding any security
issues). Come to think of it, the syscall wrappers for each method should do
that for us as well.

diff --git a/ipc/syscall.c b/ipc/syscall.c
index 355a3da..1d6f53f 100644
--- a/ipc/syscall.c
+++ b/ipc/syscall.c
@@ -13,7 +13,7 @@
 #include <linux/syscalls.h>
 #include <linux/uaccess.h>
 
-SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, int, second,
+SYSCALL_DEFINE6(ipc, unsigned int, call, int, first, unsigned long, second,
 		unsigned long, third, void __user *, ptr, long, fifth)
 {
 	int version, ret;

^ permalink raw reply related	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2010-03-22 22:01 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2010-03-22  6:47 ppc64 sys_ipc breakage in 2.6.34-rc2 Anton Blanchard
2010-03-22 18:00 ` Andreas Schwab
2010-03-22 18:07   ` Linus Torvalds
2010-03-22 19:56     ` Andreas Schwab
2010-03-22 22:01     ` Anton Blanchard
2010-03-22 20:01 ` Christoph Hellwig

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).