vm86 segfaults with NX bit

vm86 segfaults with NX bit

[Takashi Iwai]

Hi,

X guys reported that VESA driver segfaults in vm86 mode with a kernel
with CONFIG_X86_PAE, and it turned to be the NX bit.  See Novell
bugzilla #443440:
	https://bugzilla.novell.com/show_bug.cgi?id=443440

I can confirm that the latest 2.6.28-rc still causes the same problem,
and it disappears when booted with noexec=off option.

Also, when NX bit is reset for the first 1MB (or smaller) in
do_sys_vm86(), it also works as expected.  But I have little clue
where to restore the bit again.

Any good suggestions / solutions?


thanks,

Takashi

Re: vm86 segfaults with NX bit

[Alan Cox]

> Also, when NX bit is reset for the first 1MB (or smaller) in
> do_sys_vm86(), it also works as expected.  But I have little clue
> where to restore the bit again.
> 
> Any good suggestions / solutions?

Does it behave if you clear NX just for  640K-1MB, and page 0 ?

Alan

Re: vm86 segfaults with NX bit

[Takashi Iwai]

At Thu, 20 Nov 2008 15:58:08 +0000,
Alan Cox wrote:
> 
> > Also, when NX bit is reset for the first 1MB (or smaller) in
> > do_sys_vm86(), it also works as expected.  But I have little clue
> > where to restore the bit again.
> > 
> > Any good suggestions / solutions?
> 
> Does it behave if you clear NX just for  640K-1MB, and page 0 ?

Yes.


Takashi

Re: vm86 segfaults with NX bit

[Takashi Iwai]

At Thu, 20 Nov 2008 17:00:03 +0100,
I wrote:
> 
> At Thu, 20 Nov 2008 15:58:08 +0000,
> Alan Cox wrote:
> > 
> > > Also, when NX bit is reset for the first 1MB (or smaller) in
> > > do_sys_vm86(), it also works as expected.  But I have little clue
> > > where to restore the bit again.
> > > 
> > > Any good suggestions / solutions?
> > 
> > Does it behave if you clear NX just for  640K-1MB, and page 0 ?
> 
> Yes.

Ah, well, that's not exactly what I tested.  I cleared NX on all 1MB
pages like the patch below.


thanks,

Takashi

---
diff --git a/arch/x86/kernel/vm86_32.c b/arch/x86/kernel/vm86_32.c
index 4eeb5cf..60b00d0 100644
--- a/arch/x86/kernel/vm86_32.c
+++ b/arch/x86/kernel/vm86_32.c
@@ -192,6 +192,45 @@ out:
 	flush_tlb();
 }
 
+#ifdef CONFIG_X86_PAE
+static void mark_nx(struct mm_struct *mm, int set)
+{
+	pgd_t *pgd;
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	spinlock_t *ptl;
+	int i;
+
+	if (!(__supported_pte_mask & _PAGE_NX))
+		return;
+		
+	pgd = pgd_offset(mm, 0);
+	if (pgd_none_or_clear_bad(pgd))
+		goto out;
+	pud = pud_offset(pgd, 0);
+	if (pud_none_or_clear_bad(pud))
+		goto out;
+	pmd = pmd_offset(pud, 0);
+	if (pmd_none_or_clear_bad(pmd))
+		goto out;
+	pte = pte_offset_map_lock(mm, pmd, 0, &ptl);
+	for (i = 0; i < 256; i++) {
+		if (pte_present(*pte)) {
+			if (set)
+				set_pte(pte, __pte(pte_val(*pte) | _PAGE_NX));
+			else
+				set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_NX));
+		}
+		pte++;
+	}
+	pte_unmap_unlock(pte, ptl);
+out:
+	flush_tlb();
+}
+#else
+#define mark_nx(mm,x) do { } while (0)
+#endif
 
 
 static int do_vm86_irq_handling(int subfunction, int irqnumber);
@@ -335,6 +374,7 @@ static void do_sys_vm86(struct kernel_vm86_struct *info, struct task_struct *tsk
 	tsk->thread.screen_bitmap = info->screen_bitmap;
 	if (info->flags & VM86_SCREEN_BITMAP)
 		mark_screen_rdonly(tsk->mm);
+	mark_nx(tsk->mm, 0);
 
 	/*call audit_syscall_exit since we do not exit via the normal paths */
 	if (unlikely(current->audit_context))

Re: vm86 segfaults with NX bit

[Arjan van de Ven]

On Thu, 20 Nov 2008 16:39:06 +0100
Takashi Iwai <tiwai@suse.de> wrote:

> Hi,
> 
> X guys reported that VESA driver segfaults in vm86 mode with a kernel
> with CONFIG_X86_PAE, and it turned to be the NX bit.  See Novell
> bugzilla #443440:
> 	https://bugzilla.novell.com/show_bug.cgi?id=443440
> 
> I can confirm that the latest 2.6.28-rc still causes the same problem,
> and it disappears when booted with noexec=off option.
> 
> Also, when NX bit is reset for the first 1MB (or smaller) in
> do_sys_vm86(), it also works as expected.  But I have little clue
> where to restore the bit again.
> 
> Any good suggestions / solutions?
> 

did the code that mapped this memory setup use PROT_EXEC ?


-- 
Arjan van de Ven 	Intel Open Source Technology Centre
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: vm86 segfaults with NX bit

[Takashi Iwai]

At Thu, 20 Nov 2008 08:58:44 -0800,
Arjan van de Ven wrote:
> 
> On Thu, 20 Nov 2008 16:39:06 +0100
> Takashi Iwai <tiwai@suse.de> wrote:
> 
> > Hi,
> > 
> > X guys reported that VESA driver segfaults in vm86 mode with a kernel
> > with CONFIG_X86_PAE, and it turned to be the NX bit.  See Novell
> > bugzilla #443440:
> > 	https://bugzilla.novell.com/show_bug.cgi?id=443440
> > 
> > I can confirm that the latest 2.6.28-rc still causes the same problem,
> > and it disappears when booted with noexec=off option.
> > 
> > Also, when NX bit is reset for the first 1MB (or smaller) in
> > do_sys_vm86(), it also works as expected.  But I have little clue
> > where to restore the bit again.
> > 
> > Any good suggestions / solutions?
> > 
> 
> did the code that mapped this memory setup use PROT_EXEC ?

A good question...
Luc, Egbert, how does VESA driver handle it?


Takashi

Re: vm86 segfaults with NX bit

[Takashi Iwai]

At Thu, 20 Nov 2008 18:01:08 +0100,
I wrote:
> 
> At Thu, 20 Nov 2008 08:58:44 -0800,
> Arjan van de Ven wrote:
> > 
> > On Thu, 20 Nov 2008 16:39:06 +0100
> > Takashi Iwai <tiwai@suse.de> wrote:
> > 
> > > Hi,
> > > 
> > > X guys reported that VESA driver segfaults in vm86 mode with a kernel
> > > with CONFIG_X86_PAE, and it turned to be the NX bit.  See Novell
> > > bugzilla #443440:
> > > 	https://bugzilla.novell.com/show_bug.cgi?id=443440
> > > 
> > > I can confirm that the latest 2.6.28-rc still causes the same problem,
> > > and it disappears when booted with noexec=off option.
> > > 
> > > Also, when NX bit is reset for the first 1MB (or smaller) in
> > > do_sys_vm86(), it also works as expected.  But I have little clue
> > > where to restore the bit again.
> > > 
> > > Any good suggestions / solutions?
> > > 
> > 
> > did the code that mapped this memory setup use PROT_EXEC ?
> 
> A good question...
> Luc, Egbert, how does VESA driver handle it?

Just looking through the vesa directory of xorg-server code, PROT_EXEC
is passed for all mappings, at least.

Please correct me if I'm looking at a wrong place...


thanks,

Takashi

Re: vm86 segfaults with NX bit

[H. Peter Anvin]

Takashi Iwai wrote:
>>>>
>>> did the code that mapped this memory setup use PROT_EXEC ?
>> A good question...
>> Luc, Egbert, how does VESA driver handle it?
> 
> Just looking through the vesa directory of xorg-server code, PROT_EXEC
> is passed for all mappings, at least.
> 
> Please correct me if I'm looking at a wrong place...
> 

Please do an strace of the X server startup up to the point of the crash.

I am betting that PROT_EXEC is missing from an mmap(), or that it needs 
an additional mprotect().

	-hpa

Re: vm86 segfaults with NX bit

[Takashi Iwai]

At Thu, 20 Nov 2008 13:24:38 -0800,
H. Peter Anvin wrote:
> 
> Takashi Iwai wrote:
> >>>>
> >>> did the code that mapped this memory setup use PROT_EXEC ?
> >> A good question...
> >> Luc, Egbert, how does VESA driver handle it?
> > 
> > Just looking through the vesa directory of xorg-server code, PROT_EXEC
> > is passed for all mappings, at least.
> > 
> > Please correct me if I'm looking at a wrong place...
> > 
> 
> Please do an strace of the X server startup up to the point of the crash.

Well, it's not exactly a crash.  VESA driver catches SIGSEGV in its
signal handler, spews the error messages and skips the vm86
operation.  The segfault occurs during every vm86old syscall.

> I am betting that PROT_EXEC is missing from an mmap(), or that it needs 
> an additional mprotect().

PROT_EXEC seems used in every mmap.  But, VESA and int10 codes in
xorg is old and hard to follow for me.

Hopefully Egbert can give us a better insight...


Takashi

Re: vm86 segfaults with NX bit

[Egbert Eich]

[-- Attachment #1: message body text --]
[-- Type: text/plain, Size: 1614 bytes --]

Takashi Iwai writes:
 > At Thu, 20 Nov 2008 08:58:44 -0800,
 > Arjan van de Ven wrote:
 > > 
 > > On Thu, 20 Nov 2008 16:39:06 +0100
 > > Takashi Iwai <tiwai@suse.de> wrote:
 > > 
 > > > Hi,
 > > > 
 > > > X guys reported that VESA driver segfaults in vm86 mode with a kernel
 > > > with CONFIG_X86_PAE, and it turned to be the NX bit.  See Novell
 > > > bugzilla #443440:
 > > > 	https://bugzilla.novell.com/show_bug.cgi?id=443440
 > > > 
 > > > I can confirm that the latest 2.6.28-rc still causes the same problem,
 > > > and it disappears when booted with noexec=off option.
 > > > 
 > > > Also, when NX bit is reset for the first 1MB (or smaller) in
 > > > do_sys_vm86(), it also works as expected.  But I have little clue
 > > > where to restore the bit again.
 > > > 
 > > > Any good suggestions / solutions?
 > > > 
 > > 
 > > did the code that mapped this memory setup use PROT_EXEC ?
 > 
 > A good question...
 > Luc, Egbert, how does VESA driver handle it?


Yes, we set the PROT_EXEC for the VBIOS. But the segfault doesn't happen
in VBIOS as far as I can tell. It happens on the 'exit instruction', a
hlt (0xf6) which is trapped by vm86 so it returns to the caller.
This is done by putting 0xf6 into address 0x600 and putting this address
on the stack.
The memory that's mapped at 0x600 in the Xserver's address space is
allocated thru shmget() and attached to the right place in the processes
memory space for vm86.
We should call mprotect() on these ranges to set this memory executable.
Hey, this code was orginally written almost 10 years ago!

The patch below fixes the issue.

Cheers,
	Egbert.


[-- Attachment #2: diff.vm86_exec --]
[-- Type: text/plain, Size: 1414 bytes --]

diff --git a/hw/xfree86/os-support/linux/int10/linux.c b/hw/xfree86/os-support/linux/int10/linux.c
index 67eb161..b15f7fd 100644
--- a/hw/xfree86/os-support/linux/int10/linux.c
+++ b/hw/xfree86/os-support/linux/int10/linux.c
@@ -1,6 +1,6 @@
 /*
  * linux specific part of the int10 module
- * Copyright 1999, 2000, 2001, 2002, 2003, 2004 Egbert Eich
+ * Copyright 1999, 2000, 2001, 2002, 2003, 2004, 2008 Egbert Eich
  */
 #ifdef HAVE_XORG_CONFIG_H
 #include <xorg-config.h>
@@ -357,7 +357,10 @@ MapCurrentInt10(xf86Int10InfoPtr pInt)
 		   "shmat(low_mem) error: %s\n",strerror(errno));
 	return FALSE;
     }
-    
+    if (mprotect((void*)0, V_RAM, PROT_READ|PROT_WRITE|PROT_EXEC) != 0)
+        xf86DrvMsg(pInt->scrnIndex, X_ERROR,
+		   "Cannot set EXEC bit on low memory: %s\n", strerror(errno));
+
     if (((linuxInt10Priv*)pInt->private)->highMem >= 0) {
 	addr = shmat(((linuxInt10Priv*)pInt->private)->highMem,
 		     (char*)HIGH_MEM, 0);
@@ -368,6 +371,11 @@ MapCurrentInt10(xf86Int10InfoPtr pInt)
 		       "shmget error: %s\n",strerror(errno));
 	    return FALSE;
 	}
+	if (mprotect((void*)HIGH_MEM, HIGH_MEM_SIZE,
+		     PROT_READ|PROT_WRITE|PROT_EXEC) != 0)
+	    xf86DrvMsg(pInt->scrnIndex, X_ERROR,
+		       "Cannot set EXEC bit on high memory: %s\n",
+		       strerror(errno));
     } else {
 	if ((fd = open(DEV_MEM, O_RDWR, 0)) >= 0) {
 	    if (mmap((void *)(V_BIOS), SYS_BIOS - V_BIOS,

Re: vm86 segfaults with NX bit

[Takashi Iwai]

At Fri, 21 Nov 2008 18:42:04 +0100,
Egbert Eich wrote:
> 
> Takashi Iwai writes:
>  > At Thu, 20 Nov 2008 08:58:44 -0800,
>  > Arjan van de Ven wrote:
>  > > 
>  > > On Thu, 20 Nov 2008 16:39:06 +0100
>  > > Takashi Iwai <tiwai@suse.de> wrote:
>  > > 
>  > > > Hi,
>  > > > 
>  > > > X guys reported that VESA driver segfaults in vm86 mode with a kernel
>  > > > with CONFIG_X86_PAE, and it turned to be the NX bit.  See Novell
>  > > > bugzilla #443440:
>  > > > 	https://bugzilla.novell.com/show_bug.cgi?id=443440
>  > > > 
>  > > > I can confirm that the latest 2.6.28-rc still causes the same problem,
>  > > > and it disappears when booted with noexec=off option.
>  > > > 
>  > > > Also, when NX bit is reset for the first 1MB (or smaller) in
>  > > > do_sys_vm86(), it also works as expected.  But I have little clue
>  > > > where to restore the bit again.
>  > > > 
>  > > > Any good suggestions / solutions?
>  > > > 
>  > > 
>  > > did the code that mapped this memory setup use PROT_EXEC ?
>  > 
>  > A good question...
>  > Luc, Egbert, how does VESA driver handle it?
> 
> 
> Yes, we set the PROT_EXEC for the VBIOS. But the segfault doesn't happen
> in VBIOS as far as I can tell. It happens on the 'exit instruction', a
> hlt (0xf6) which is trapped by vm86 so it returns to the caller.
> This is done by putting 0xf6 into address 0x600 and putting this address
> on the stack.
> The memory that's mapped at 0x600 in the Xserver's address space is
> allocated thru shmget() and attached to the right place in the processes
> memory space for vm86.
> We should call mprotect() on these ranges to set this memory executable.
> Hey, this code was orginally written almost 10 years ago!
> 
> The patch below fixes the issue.

Good to know that a die-hard bug is fixed now.
Thanks!


Takashi