mapping of memory below 16Mb

mapping of memory below 16Mb

[Jan Beulich]

One point I got a little surprised about while doing the 1Gb page support:
The 1:1 mapping of the low 16 Mb (and most significantly the first Mb)
created at boot time is being retained permanently. Shouldn't Xen be
following the E820 map also for this memory range (by either not
mapping non-RAM at all or at best mapping them UC/WP)?

The WP attribute raises another question: Are there any plans to make
Xen use PAT rather than (relying on) MTRRs? Linux folks seem to be
finally getting closer to this, too, so even virtual PAT support may be
needed in not too distant future...

Jan

Re: mapping of memory below 16Mb

[Keir Fraser]

On 24/1/08 15:15, "Jan Beulich" <jbeulich@novell.com> wrote:

> One point I got a little surprised about while doing the 1Gb page support:
> The 1:1 mapping of the low 16 Mb (and most significantly the first Mb)
> created at boot time is being retained permanently. Shouldn't Xen be
> following the E820 map also for this memory range (by either not
> mapping non-RAM at all or at best mapping them UC/WP)?

Below 1MB is a special case, and 1MB-16MB we're dodging BIOS tables, not
actual hardware. At least this is the case for all system I know about. If
there are exceptions which we care about, then we can try harder. Otherwise
that code is quite complicated enough as it is.

> The WP attribute raises another question: Are there any plans to make
> Xen use PAT rather than (relying on) MTRRs? Linux folks seem to be
> finally getting closer to this, too, so even virtual PAT support may be
> needed in not too distant future...

We already support PAT for both PV guests and HVM guests.

 -- Keir

> Jan
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: mapping of memory below 16Mb

[Jan Beulich]

>> The WP attribute raises another question: Are there any plans to make
>> Xen use PAT rather than (relying on) MTRRs? Linux folks seem to be
>> finally getting closer to this, too, so even virtual PAT support may be
>> needed in not too distant future...
>
>We already support PAT for both PV guests and HVM guests.

Hmm, I must have missed this going in. Is the indexing into PAT then
considered part of the ABI (as I can't see either a hypercall a guest
could use to learn what bit combination to use for certain attributes or
one to set a virtual PAT value)?

And regardless of that, SVM code has no sign of handling accesses to the
PAT MSR.

Jan

Re: mapping of memory below 16Mb

[Keir Fraser]

On 24/1/08 15:57, "Jan Beulich" <jbeulich@novell.com> wrote:

>>> The WP attribute raises another question: Are there any plans to make
>>> Xen use PAT rather than (relying on) MTRRs? Linux folks seem to be
>>> finally getting closer to this, too, so even virtual PAT support may be
>>> needed in not too distant future...
>> 
>> We already support PAT for both PV guests and HVM guests.
> 
> Hmm, I must have missed this going in. Is the indexing into PAT then
> considered part of the ABI (as I can't see either a hypercall a guest
> could use to learn what bit combination to use for certain attributes or
> one to set a virtual PAT value)?

Xen statically initialises the PAT MSR to 0x0000050100070406 and that is
part of the PV guest ABI. PAT6 and PAT7 indexes are reserved -- their memory
type may change in future.

> And regardless of that, SVM code has no sign of handling accesses to the
> PAT MSR.

Oh, some VMX code needs to be common-ised then. But since this only makes
any difference for guests with direct hardware passthru, and that doesn't
work on AMD systems yet, this is hardly a priority.

 -- Keir

Re: mapping of memory below 16Mb

[Keir Fraser]

On 24/1/08 15:26, "Keir Fraser" <Keir.Fraser@cl.cam.ac.uk> wrote:

>> One point I got a little surprised about while doing the 1Gb page support:
>> The 1:1 mapping of the low 16 Mb (and most significantly the first Mb)
>> created at boot time is being retained permanently. Shouldn't Xen be
>> following the E820 map also for this memory range (by either not
>> mapping non-RAM at all or at best mapping them UC/WP)?
> 
> Below 1MB is a special case, and 1MB-16MB we're dodging BIOS tables, not
> actual hardware. At least this is the case for all system I know about. If
> there are exceptions which we care about, then we can try harder. Otherwise
> that code is quite complicated enough as it is.

Actually this isn't hard to fix now I think about it. Also we should be
mapping the range 0-4MB or 0-2MB with 4kB mappings anyway, to avoid
conflicting with memory access constraints specified by the fixed MTRRs.
Every modern BIOS sets the fixed MTRRs with a mixture of memory types, and
if we map that range with a superpage we either get undefined behaviour or
at best (e.g., Intel CPUs) the processor will implicitly shatter the
superpage mapping so that we get 4kB-mapping levels of performance anyway.

This has a side 'benefit' that the range 0-1GB cannot be mapped with a 1GB
superpage mapping, and that simplifies your patch a bit (since we will never
need to free the statically-allocated first few l2e ident mapping pages).

 -- Keir

Re: mapping of memory below 16Mb

[Jan Beulich]

>> The WP attribute raises another question: Are there any plans to make
>> Xen use PAT rather than (relying on) MTRRs? Linux folks seem to be
>> finally getting closer to this, too, so even virtual PAT support may be
>> needed in not too distant future...
>
>We already support PAT for both PV guests and HVM guests.

Oh, and my understanding of PAT support goes beyond just setting the
PAT MSR and adjusting page table flags. Specifically, if we have PAT we
should never need to modify MTRRs, which would eliminate conflicts on
updates of those that I'm seeing on various boxes, always just
depending on how BIOS sets them up, and always related to overlaps
of the video memory with existing MTRR ranges.

Jan

Re: mapping of memory below 16Mb

[Keir Fraser]

On 24/1/08 16:24, "Jan Beulich" <jbeulich@novell.com> wrote:

>>> The WP attribute raises another question: Are there any plans to make
>>> Xen use PAT rather than (relying on) MTRRs? Linux folks seem to be
>>> finally getting closer to this, too, so even virtual PAT support may be
>>> needed in not too distant future...
>> 
>> We already support PAT for both PV guests and HVM guests.
> 
> Oh, and my understanding of PAT support goes beyond just setting the
> PAT MSR and adjusting page table flags. Specifically, if we have PAT we
> should never need to modify MTRRs, which would eliminate conflicts on
> updates of those that I'm seeing on various boxes, always just
> depending on how BIOS sets them up, and always related to overlaps
> of the video memory with existing MTRR ranges.

That's a dom0 issue. Xen itself does not proactively mess with the MTRRs.

 -- Keir

Re: mapping of memory below 16Mb

[Jan Beulich]

>>> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> 24.01.08 17:27 >>>
>On 24/1/08 16:24, "Jan Beulich" <jbeulich@novell.com> wrote:
>
>>>> The WP attribute raises another question: Are there any plans to make
>>>> Xen use PAT rather than (relying on) MTRRs? Linux folks seem to be
>>>> finally getting closer to this, too, so even virtual PAT support may be
>>>> needed in not too distant future...
>>> 
>>> We already support PAT for both PV guests and HVM guests.
>> 
>> Oh, and my understanding of PAT support goes beyond just setting the
>> PAT MSR and adjusting page table flags. Specifically, if we have PAT we
>> should never need to modify MTRRs, which would eliminate conflicts on
>> updates of those that I'm seeing on various boxes, always just
>> depending on how BIOS sets them up, and always related to overlaps
>> of the video memory with existing MTRR ranges.
>
>That's a dom0 issue. Xen itself does not proactively mess with the MTRRs.

Yes and no: Yes in the sense that Dom0 could be given an alternative
hypercall to specify memory types. No in the sense that we already have
such a hypercall (setting the MTRRs) that would simply need to do
translation to PAT semantics rather than writing to the MTRR MSRs.

Jan

Re: mapping of memory below 16Mb

[Keir Fraser]

On 24/1/08 16:35, "Jan Beulich" <jbeulich@novell.com> wrote:

>> That's a dom0 issue. Xen itself does not proactively mess with the MTRRs.
> 
> Yes and no: Yes in the sense that Dom0 could be given an alternative
> hypercall to specify memory types. No in the sense that we already have
> such a hypercall (setting the MTRRs) that would simply need to do
> translation to PAT semantics rather than writing to the MTRR MSRs.

If Linux is getting native PAT support, then we shouldn't need MTRR->PAT
shimming code in Xen. And thank goodness for that!

 -- Keir

Re: mapping of memory below 16Mb

[Jan Beulich]

>>> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> 24.01.08 17:22 >>>
>On 24/1/08 15:26, "Keir Fraser" <Keir.Fraser@cl.cam.ac.uk> wrote:
>
>>> One point I got a little surprised about while doing the 1Gb page support:
>>> The 1:1 mapping of the low 16 Mb (and most significantly the first Mb)
>>> created at boot time is being retained permanently. Shouldn't Xen be
>>> following the E820 map also for this memory range (by either not
>>> mapping non-RAM at all or at best mapping them UC/WP)?
>> 
>> Below 1MB is a special case, and 1MB-16MB we're dodging BIOS tables, not
>> actual hardware. At least this is the case for all system I know about. If
>> there are exceptions which we care about, then we can try harder. Otherwise
>> that code is quite complicated enough as it is.
>
>Actually this isn't hard to fix now I think about it. Also we should be
>mapping the range 0-4MB or 0-2MB with 4kB mappings anyway, to avoid
>conflicting with memory access constraints specified by the fixed MTRRs.
>Every modern BIOS sets the fixed MTRRs with a mixture of memory types, and
>if we map that range with a superpage we either get undefined behaviour or
>at best (e.g., Intel CPUs) the processor will implicitly shatter the
>superpage mapping so that we get 4kB-mapping levels of performance anyway.
>
>This has a side 'benefit' that the range 0-1GB cannot be mapped with a 1GB
>superpage mapping, and that simplifies your patch a bit (since we will never
>need to free the statically-allocated first few l2e ident mapping pages).

Actually - no, it would guarantee only the first of them would never be
freed. The others, as long as head.S puts them into l3_identmap
unconditionally, will still be subject to eventual freeing.

Jan

Re: mapping of memory below 16Mb

[Keir Fraser]

On 25/1/08 07:51, "Jan Beulich" <jbeulich@novell.com> wrote:

>> This has a side 'benefit' that the range 0-1GB cannot be mapped with a 1GB
>> superpage mapping, and that simplifies your patch a bit (since we will never
>> need to free the statically-allocated first few l2e ident mapping pages).
> 
> Actually - no, it would guarantee only the first of them would never be
> freed. The others, as long as head.S puts them into l3_identmap
> unconditionally, will still be subject to eventual freeing.

Ah yes, because they map a 4GB range.

 -- Keir