From: "Antonino A. Daplas" <adaplas@gmail.com>
To: Tero Roponen <teanropo@jyu.fi>
Cc: Linus Torvalds <torvalds@linux-foundation.org>,
Michal Piotrowski <michal.k.k.piotrowski@gmail.com>,
Andrew Morton <akpm@linux-foundation.org>,
LKML <linux-kernel@vger.kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
David Miller <davem@davemloft.net>
Subject: [PATCH] neofb: Fix pseudo_palette array overrun in neofb_setcolreg
Date: Thu, 31 May 2007 14:04:57 +0800 [thread overview]
Message-ID: <465E6589.5040903@gmail.com> (raw)
In-Reply-To: <Pine.LNX.4.64.0705310852400.8734@jalava.cc.jyu.fi>
The pseudo_palette has room for 16 entries only, but in truecolor mode, it
attempts to write 256.
Signed-off-by: Antonino Daplas <adaplas@gmail.com>
Acked-by: Tero Roponen <teanropo@jyu.fi>
---
Tero Roponen wrote:
> On Thu, 31 May 2007, Antonino A. Daplas wrote:
>
>> On Wed, 2007-05-30 at 19:33 -0700, Linus Torvalds wrote:
>>> On Tue, 29 May 2007, Michal Piotrowski wrote:
>>>> TTY
>>>>
>>>> Subject : tty-related oops in latest kernel(s)
>>>> References : http://lkml.org/lkml/2007/5/27/104
>>>> Submitter : Tero Roponen <teanropo@jyu.fi>
>>>> Status : problem is being debugged
>>> People seem to have debugged this to neofb palette handling, but I haven't
>>> seen a patch. Antonino?
>>>
>> Already posted one for testing. I'm waiting for Tero to confirm.
>>
>> Tony
>>
>
> Ok, I tested all the cases I have reported: no corruption
> and nothing in slabinfo -v. This seems to be the right fix.
>
Okay, thanks for testing.
Tony
drivers/video/neofb.c | 30 ++++++++++++++++--------------
1 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/drivers/video/neofb.c b/drivers/video/neofb.c
index bd30aba..731d7a5 100644
--- a/drivers/video/neofb.c
+++ b/drivers/video/neofb.c
@@ -1286,34 +1286,36 @@ static int neofb_setcolreg(u_int regno,
if (regno >= fb->cmap.len || regno > 255)
return -EINVAL;
- switch (fb->var.bits_per_pixel) {
- case 8:
+ if (fb->var.bits_per_pixel <= 8) {
outb(regno, 0x3c8);
outb(red >> 10, 0x3c9);
outb(green >> 10, 0x3c9);
outb(blue >> 10, 0x3c9);
- break;
- case 16:
- ((u32 *) fb->pseudo_palette)[regno] =
+ } else if (regno < 16) {
+ switch (fb->var.bits_per_pixel) {
+ case 16:
+ ((u32 *) fb->pseudo_palette)[regno] =
((red & 0xf800)) | ((green & 0xfc00) >> 5) |
((blue & 0xf800) >> 11);
- break;
- case 24:
- ((u32 *) fb->pseudo_palette)[regno] =
+ break;
+ case 24:
+ ((u32 *) fb->pseudo_palette)[regno] =
((red & 0xff00) << 8) | ((green & 0xff00)) |
((blue & 0xff00) >> 8);
- break;
+ break;
#ifdef NO_32BIT_SUPPORT_YET
- case 32:
- ((u32 *) fb->pseudo_palette)[regno] =
+ case 32:
+ ((u32 *) fb->pseudo_palette)[regno] =
((transp & 0xff00) << 16) | ((red & 0xff00) << 8) |
((green & 0xff00)) | ((blue & 0xff00) >> 8);
- break;
+ break;
#endif
- default:
- return 1;
+ default:
+ return 1;
+ }
}
+
return 0;
}
next prev parent reply other threads:[~2007-05-31 6:05 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <465C188F.9000900@googlemail.com>
2007-05-29 12:52 ` [3/4] 2.6.22-rc3: known regressions Michal Piotrowski
2007-05-29 21:47 ` [PATCH] NOHZ: prevent multiplication overflow - stop timer for huge timeouts Thomas Gleixner
2007-05-29 23:02 ` David Miller
2007-05-31 2:33 ` [3/4] 2.6.22-rc3: known regressions Linus Torvalds
2007-05-31 4:51 ` Antonino A. Daplas
2007-05-31 5:54 ` Tero Roponen
2007-05-31 6:04 ` Antonino A. Daplas [this message]
2007-06-05 11:09 ` [PATCH] neofb: Fix pseudo_palette array overrun in neofb_setcolreg Antonino A. Daplas
2007-06-05 11:34 ` [PATCH] [RESEND] " Antonino A. Daplas
2007-05-29 12:53 ` [4/4] 2.6.22-rc3: known regressions Michal Piotrowski
2007-06-09 11:38 ` Mauro Carvalho Chehab
2007-05-29 12:56 ` [2/4] " Michal Piotrowski
2007-05-29 15:01 ` Stephen Hemminger
2007-05-31 2:22 ` Linus Torvalds
2007-05-31 22:08 ` Mike Miller (OS Dev)
2007-05-31 22:22 ` Linus Torvalds
2007-05-31 22:39 ` Mike Miller (OS Dev)
2007-05-31 22:50 ` Andrew Morton
2007-05-31 23:12 ` Linus Torvalds
2007-05-31 23:17 ` Roland Dreier
2007-05-31 23:34 ` Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=465E6589.5040903@gmail.com \
--to=adaplas@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=davem@davemloft.net \
--cc=linux-kernel@vger.kernel.org \
--cc=michal.k.k.piotrowski@gmail.com \
--cc=teanropo@jyu.fi \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox