RE: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

RE: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Nakajima, Jun]

Keir Fraser wrote:
> On 18 Aug 2005, at 05:29, Nakajima, Jun wrote:
> 
>> I don't have particular preference, but the latter one's change is
>> smaller, and it can fix other places (that are using the attrributes
>> derived from __PAGE_KERNEL). But the former one is cleaner to me. If
>> people agree, I can make such a patch.
> 
> How does native x86/64 Linux avoid this problem? It doesn't mask
> __PAGE_KERNEL with __supported_pte_mask, so I'm reluctant to take this
> patch. Seems to me there is some other underlying difference between
> us and native that is not best solved by diverging even further.
> 
> Unless native doesn;t boot on these failing boxes either?
> 
>   -- Keir

One obvious reason is: xen-specific drivers are using __PAGE_KERNEL (and
they don't exist in the native Linux). If xen_contig_memory, for
example, does not work, it won't boot... Fixing xen_contig_memory did
not fix the booting problem as long as I remember? Since then we fixed
some attribute problems in other xen-specific drivers when we brough up
domU, so I'll take a look.

xen_contig_memory
...
    /* 3. Map the new extent in place of old pages. */
    for (i = 0; i < (1<<order); i++) {
        BUG_ON(HYPERVISOR_update_va_mapping(
            vstart + (i*PAGE_SIZE),
            __pte_ma(((mfn+i)<<PAGE_SHIFT)|__PAGE_KERNEL), 0));
        xen_machphys_update(mfn+i, (__pa(vstart)>>PAGE_SHIFT)+i);
        phys_to_machine_mapping[(__pa(vstart)>>PAGE_SHIFT)+i] = mfn+i;
    }


blkback.c:  dispatch_probe
static void dispatch_probe(blkif_t *blkif, blkif_request_t *req)
{
 ...
#ifdef CONFIG_XEN_BLKDEV_TAP_BE
    if ( HYPERVISOR_update_va_mapping_otherdomain(
        MMAP_VADDR(pending_idx, 0),
        (pte_t) { (req->frame_and_sects[0] & PAGE_MASK) | __PAGE_KERNEL
},
        0, (blkif->is_blktap ? ID_TO_DOM(req->id) : blkif->domid) ) )

        goto out;
#else
    if ( HYPERVISOR_update_va_mapping_otherdomain(
        MMAP_VADDR(pending_idx, 0),
        (pte_t) { (req->frame_and_sects[0] & PAGE_MASK) | __PAGE_KERNEL
},
        0, blkif->domid) )

        goto out;
#endif
#endif /* endif CONFIG_XEN_BLKDEV_GRANT */

Jun
---
Intel Open Source Technology Center

Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Keir Fraser]

On 18 Aug 2005, at 15:15, Nakajima, Jun wrote:

>
> One obvious reason is: xen-specific drivers are using __PAGE_KERNEL 
> (and
> they don't exist in the native Linux). If xen_contig_memory, for
> example, does not work, it won't boot... Fixing xen_contig_memory did
> not fix the booting problem as long as I remember?

Yeah, I see what you mean. I've just applied a patch that cleans up the 
few remaining bad spots where we weren't using pfn_pte() or 
pfn_pte_ma() constructors. I can now boot okay even with NX/XD disabled 
in Xen.

By the way -- from what I've seen of this NX/XD discussion so far, it 
sounds like even if CPUID says that NX/XD is supported, and you set the 
EFER.NXE bit, the BIOS can still cause this to be ignored! Is this true 
and, if so, what will happen if you try to use NX/XD anyway? i.e., will 
the bit be ignored, or will its use cause a page fault?

  -- Keir

Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Jerone Young]

On Thu, 2005-08-18 at 16:09 +0100, Keir Fraser wrote:
> On 18 Aug 2005, at 15:15, Nakajima, Jun wrote:
> 
> >
> > One obvious reason is: xen-specific drivers are using __PAGE_KERNEL 
> > (and
> > they don't exist in the native Linux). If xen_contig_memory, for
> > example, does not work, it won't boot... Fixing xen_contig_memory did
> > not fix the booting problem as long as I remember?
> 
> Yeah, I see what you mean. I've just applied a patch that cleans up the 
> few remaining bad spots where we weren't using pfn_pte() or 
> pfn_pte_ma() constructors. I can now boot okay even with NX/XD disabled 
> in Xen.
> 
> By the way -- from what I've seen of this NX/XD discussion so far, it 
> sounds like even if CPUID says that NX/XD is supported, and you set the 
> EFER.NXE bit, the BIOS can still cause this to be ignored!

If I remember correctly (will create a patch shortly to make 100% sure).
If the option is off in the BIOS than NX bit will not show up as a CPU
feature flag. And of course once it's on in the bios then it does.

>  Is this true 
> and, if so, what will happen if you try to use NX/XD anyway? i.e., will 
> the bit be ignored, or will its use cause a page fault?
> 
>   -- Keir
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
-- 
Jerone Young
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)

Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Keir Fraser]

On 18 Aug 2005, at 16:39, Jerone Young wrote:

> If I remember correctly (will create a patch shortly to make 100% 
> sure).
> If the option is off in the BIOS than NX bit will not show up as a CPU
> feature flag. And of course once it's on in the bios then it does.

Good to hear. Case closed on nx/xd then...

  -- Keir

Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Andi Kleen]

Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> 
> By the way -- from what I've seen of this NX/XD discussion so far, it
> sounds like even if CPUID says that NX/XD is supported, and you set
> the EFER.NXE bit, the BIOS can still cause this to be ignored! Is this
> true and, if so, what will happen if you try to use NX/XD anyway?
> i.e., will the bit be ignored, or will its use cause a page fault?

I don't think that can happen. When NX is disabled or not there it
doesn't appear in CPUID.

-Andi

Re: Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Jerone Young]

On Thu, 2005-08-18 at 17:51 +0200, Andi Kleen wrote:
> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> > 
> > By the way -- from what I've seen of this NX/XD discussion so far, it
> > sounds like even if CPUID says that NX/XD is supported, and you set
> > the EFER.NXE bit, the BIOS can still cause this to be ignored! Is this
> > true and, if so, what will happen if you try to use NX/XD anyway?
> > i.e., will the bit be ignored, or will its use cause a page fault?
> 
> I don't think that can happen. When NX is disabled or not there it
> doesn't appear in CPUID.

Ah but it does:

With NX bit cut "on" in bios:
/proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      :                   Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 1
cpu MHz         : 800.028
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
nx lm constant_tsc pni monitor ds_cpl cid cx16 xtpr   <--- NX found here
bogomips        : 6396.31
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:

With NX bit cut "off" in bios:
cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      :                   Intel(R) Xeon(TM) CPU 3.20GHz
stepping        : 1
cpu MHz         : 3200.280
cache size      : 1024 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall
lm pni monitor ds_cpl cid cmpxchg16b
bogomips        : 6324.22
clflush size    : 64
cache_alignment : 128
address sizes   : 36 bits physical, 48 bits virtual
power management:

Notice no nx in the flags when NX is turned off in the bios. 




> 
> -Andi
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
-- 
Jerone Young
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)

Re: Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Andi Kleen]

On Thu, Aug 18, 2005 at 11:05:42AM -0500, Jerone Young wrote:
> On Thu, 2005-08-18 at 17:51 +0200, Andi Kleen wrote:
> > Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> > > 
> > > By the way -- from what I've seen of this NX/XD discussion so far, it
> > > sounds like even if CPUID says that NX/XD is supported, and you set
> > > the EFER.NXE bit, the BIOS can still cause this to be ignored! Is this
> > > true and, if so, what will happen if you try to use NX/XD anyway?
> > > i.e., will the bit be ignored, or will its use cause a page fault?
> > 
> > I don't think that can happen. When NX is disabled or not there it
> > doesn't appear in CPUID.
> 
> Ah but it does:

Your example confirms what I said.

-Andi

Re: Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Jerone Young]

On Thu, 2005-08-18 at 18:18 +0200, Andi Kleen wrote:
> On Thu, Aug 18, 2005 at 11:05:42AM -0500, Jerone Young wrote:
> > On Thu, 2005-08-18 at 17:51 +0200, Andi Kleen wrote:
> > > Keir Fraser <Keir.Fraser@cl.cam.ac.uk> writes:
> > > > 
> > > > By the way -- from what I've seen of this NX/XD discussion so far, it
> > > > sounds like even if CPUID says that NX/XD is supported, and you set
> > > > the EFER.NXE bit, the BIOS can still cause this to be ignored! Is this
> > > > true and, if so, what will happen if you try to use NX/XD anyway?
> > > > i.e., will the bit be ignored, or will its use cause a page fault?
> > > 
> > > I don't think that can happen. When NX is disabled or not there it
> > > doesn't appear in CPUID.
> > 
> > Ah but it does:

hehe... I didn't realize I left that statement in there...I just wanted
to post to the list to confirm with an example. Don't worry Andi ...
your still the x86-64 god in my book!

> 
> Your example confirms what I said.

> 
> -Andi
> 
-- 
Jerone Young
IBM Linux Technology Center
jyoung5@us.ibm.com
512-838-1157 (T/L: 678-1157)

RE: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Nakajima, Jun]

Jerone Young wrote:
> This is identical to the patch Scott Parish sent awhile back. Jun you
> said that this was not a good way of going about fixing this (actually
> it was kind of toward another problem ...but I think we were also
> trying to get x86-64 xen to just boot at the time). I guess this is
> the best way then?
As far as I remember, it was not root caused at that time, then I was
not sure it was the right fix... 

The problem is that __PAGE_KERNEL is used by several places, such as
ioremap_change_attr
xen_contig_memory
blkback.c:  dispatch_probe

And __PAGE_KERNEL on native x86_64 Linux has always NX on:
#define __PAGE_KERNEL \
        (_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED |
_PAGE_NX)

So, actually we have two options to fix this:
1. Change __PAGE_KERNEL => _KERNPG_TABLE
2. The above (i.e. & __supported_pte_mask)

I don't have particular preference, but the latter one's change is
smaller, and it can fix other places (that are using the attrributes
derived from __PAGE_KERNEL). But the former one is cleaner to me. If
people agree, I can make such a patch.

Jun
---
Intel Open Source Technology Center

Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Keir Fraser]

On 18 Aug 2005, at 05:29, Nakajima, Jun wrote:

> I don't have particular preference, but the latter one's change is
> smaller, and it can fix other places (that are using the attrributes
> derived from __PAGE_KERNEL). But the former one is cleaner to me. If
> people agree, I can make such a patch.

How does native x86/64 Linux avoid this problem? It doesn't mask 
__PAGE_KERNEL with __supported_pte_mask, so I'm reluctant to take this 
patch. Seems to me there is some other underlying difference between us 
and native that is not best solved by diverging even further.

Unless native doesn;t boot on these failing boxes either?

  -- Keir

[PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Nakajima, Jun]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

I remember some people complained that x86_64 xenlinux failed to boot on
the systems with processor(s) that does not NX/XD. The patch should fix
the problem as far as I tested. 

Signed-off-by: Jun Nakajima <jun.nakajima@intel.com>

Jun
---
Intel Open Source Technology Center 

[-- Attachment #2: non_NX.patch --]
[-- Type: application/octet-stream, Size: 1048 bytes --]

# HG changeset patch
# User "Jun Nakajima <jun.nakajima@intel.com>"
# Node ID 85be353bb88cfd02d2d4f26b0642c30e8f39d635
# Parent  60da204cb23a5ac709ef46bd9092c1d6680274b1
Supporting processors that don't support NX/XD.

diff -r 60da204cb23a -r 85be353bb88c linux-2.6-xen-sparse/include/asm-xen/asm-x86_64/pgtable.h
--- a/linux-2.6-xen-sparse/include/asm-xen/asm-x86_64/pgtable.h	Wed Aug 17 20:55:55 2005
+++ b/linux-2.6-xen-sparse/include/asm-xen/asm-x86_64/pgtable.h	Wed Aug 17 22:47:14 2005
@@ -203,7 +203,8 @@
 #define PAGE_READONLY	__pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED | _PAGE_NX)
 #define PAGE_READONLY_EXEC __pgprot(_PAGE_PRESENT | _PAGE_USER | _PAGE_ACCESSED)
 #define __PAGE_KERNEL \
-	(_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_NX | _PAGE_USER )
+	((_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_NX \
+      | _PAGE_USER ) & __supported_pte_mask)
 #define __PAGE_KERNEL_EXEC \
 	(_PAGE_PRESENT | _PAGE_RW | _PAGE_DIRTY | _PAGE_ACCESSED | _PAGE_USER )
 #define __PAGE_KERNEL_NOCACHE \

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[David Hopwood]

Nakajima, Jun wrote:
> I remember some people complained that x86_64 xenlinux failed to boot on
> the systems with processor(s) that does not NX/XD. The patch should fix
> the problem as far as I tested.

Should there be a warning if NX/XD is unsupported? If it isn't because of
a BIOS option, the user probably wants to know in order to turn on the option.

-- 
David Hopwood <david.nospam.hopwood@blueyonder.co.uk>

Re: [PATCH] supporting non-NX/XD capable processors on x86_64 xenlinux

[Jerone Young]

This is identical to the patch Scott Parish sent awhile back. Jun you
said that this was not a good way of going about fixing this (actually
it was kind of toward another problem ...but I think we were also
trying to get x86-64 xen to just boot at the time). I guess this is
the best way then?

On 8/17/05, Nakajima, Jun <jun.nakajima@intel.com> wrote:
> I remember some people complained that x86_64 xenlinux failed to boot on
> the systems with processor(s) that does not NX/XD. The patch should fix
> the problem as far as I tested.
> 
> Signed-off-by: Jun Nakajima <jun.nakajima@intel.com>
> 
> Jun
> ---
> Intel Open Source Technology Center
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
> 
> 
>