From: Andi Kleen <ak@suse.de>
To: akpm@osdl.org, linux-arch@vger.kernel.org
Cc: corryk@us.ibm.com
Subject: [PATCH] Fix compat shmget overflow
Date: Wed, 9 Feb 2005 10:44:28 +0100 [thread overview]
Message-ID: <20050209094428.GB23943@wotan.suse.de> (raw)
This fixes an incorrect sign extension in the compat layer that
breaks 32bit shmget that are >2GB. sys_shmget has a signed size_t size
argument, and the int size argument comming from 32bit user space would get
sign extended to 64bit, which is wrong.
I fixed it on all compat architectures, except PPC64 which was already
ok.
It was originally debugged and fixed by Karl Rister @ IBM for SLES9 on x86-64.
Signed-off-by: Andi Kleen <ak@suse.de>
diff -u linux-2.6.11rc3/arch/sparc64/kernel/sys_sparc32.c-X linux-2.6.11rc3/arch/sparc64/kernel/sys_sparc32.c
--- linux-2.6.11rc3/arch/sparc64/kernel/sys_sparc32.c-X 2005-02-04 09:12:50.000000000 +0100
+++ linux-2.6.11rc3/arch/sparc64/kernel/sys_sparc32.c 2005-02-09 10:39:52.000000000 +0100
@@ -835,7 +835,7 @@
err = sys_shmdt(ptr);
goto out;
case SHMGET:
- err = sys_shmget(first, second, third);
+ err = sys_shmget(first, (unsigned)second, third);
goto out;
case SHMCTL:
err = do_sys32_shmctl(first, second, ptr);
diff -u linux-2.6.11rc3/arch/ia64/ia32/sys_ia32.c-X linux-2.6.11rc3/arch/ia64/ia32/sys_ia32.c
--- linux-2.6.11rc3/arch/ia64/ia32/sys_ia32.c-X 2005-02-04 09:12:42.000000000 +0100
+++ linux-2.6.11rc3/arch/ia64/ia32/sys_ia32.c 2005-02-09 10:39:03.000000000 +0100
@@ -1415,7 +1415,7 @@
case SHMDT:
return sys_shmdt(compat_ptr(ptr));
case SHMGET:
- return sys_shmget(first, second, third);
+ return sys_shmget(first, (unsigned)second, third);
case SHMCTL:
return compat_sys_shmctl(first, second, compat_ptr(ptr));
diff -u linux-2.6.11rc3/arch/mips/kernel/linux32.c-X linux-2.6.11rc3/arch/mips/kernel/linux32.c
--- linux-2.6.11rc3/arch/mips/kernel/linux32.c-X 2005-02-04 09:12:45.000000000 +0100
+++ linux-2.6.11rc3/arch/mips/kernel/linux32.c 2005-02-09 10:39:03.000000000 +0100
@@ -1115,7 +1115,7 @@
err = sys_shmdt ((char *)A(ptr));
break;
case SHMGET:
- err = sys_shmget (first, second, third);
+ err = sys_shmget (first, (unsigned)second, third);
break;
case SHMCTL:
err = do_sys32_shmctl (first, second, (void *)AA(ptr));
diff -u linux-2.6.11rc3/arch/x86_64/ia32/ipc32.c-X linux-2.6.11rc3/arch/x86_64/ia32/ipc32.c
--- linux-2.6.11rc3/arch/x86_64/ia32/ipc32.c-X 2004-04-06 13:12:04.000000000 +0200
+++ linux-2.6.11rc3/arch/x86_64/ia32/ipc32.c 2005-02-09 10:39:03.000000000 +0100
@@ -49,7 +49,7 @@
case SHMDT:
return sys_shmdt(compat_ptr(ptr));
case SHMGET:
- return sys_shmget(first, second, third);
+ return sys_shmget(first, (unsigned)second, third);
case SHMCTL:
return compat_sys_shmctl(first, second, compat_ptr(ptr));
}
diff -u linux-2.6.11rc3/arch/s390/kernel/compat_linux.c-X linux-2.6.11rc3/arch/s390/kernel/compat_linux.c
--- linux-2.6.11rc3/arch/s390/kernel/compat_linux.c-X 2005-02-04 09:12:49.000000000 +0100
+++ linux-2.6.11rc3/arch/s390/kernel/compat_linux.c 2005-02-09 10:39:28.000000000 +0100
@@ -331,7 +331,7 @@
case SHMDT:
return sys_shmdt(compat_ptr(ptr));
case SHMGET:
- return sys_shmget(first, second, third);
+ return sys_shmget(first, (unsigned)second, third);
case SHMCTL:
return compat_sys_shmctl(first, second, compat_ptr(ptr));
}
next reply other threads:[~2005-02-09 9:44 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-02-09 9:44 Andi Kleen [this message]
2005-02-09 10:08 ` [PATCH] Fix compat shmget overflow Martin Schwidefsky
2005-02-09 10:31 ` Andi Kleen
2005-02-09 12:20 ` Martin Schwidefsky
2005-02-09 12:54 ` Ralf Baechle
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20050209094428.GB23943@wotan.suse.de \
--to=ak@suse.de \
--cc=akpm@osdl.org \
--cc=corryk@us.ibm.com \
--cc=linux-arch@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox