Question about MMUEXT_PIN_L3_TABLE

Question about MMUEXT_PIN_L3_TABLE

[W. Michael Petullo]

I am trying to port Ethos, a security research kernel, to Xen on
x86+PAE. Ethos currently works fine under Xen when PAE is disabled. My
problem has to do with setting up the page tables for the first user-mode
process when the processor is in PAE mode. 

The kernel sets up its page tables and then creates an L3 page directory
pointer table (one page) for the user process. Next, the kernel
copies its L3 entry into the user process L3 table (kernel memory
starts at 0xc0000000) and zeroes out the three other entries. The
kernel then tries to pin the table using HYPERVISOR_mmuext_op() /
MMUEXT_PIN_L3_TABLE. Pinning fails, with Xen logging the following:

(XEN) mm.c:1185:d13 PAE L3 slot is shared
(XEN) mm.c:1393:d13 Failure in alloc_l3_table: entry 4
(XEN) mm.c:2070:d13 Error while validating mfn 79f0a (pfn 4c2) for type 60000000: caf=80000003 taf=60000001
(XEN) mm.c:2700:d13 Error while pinning mfn 79f0a

I have not yet had any luck finding the cause of this. From what I
can tell, the kernel's page tables are valid. The kernel initializes
memory and performs many other initializations successfully before this
point. Does anyone have any suggestions?

-- 
Mike

:wq

Re: Question about MMUEXT_PIN_L3_TABLE

[Keir Fraser]

On 08/07/2010 11:57, "W. Michael Petullo" <mike@flyn.org> wrote:

> (XEN) mm.c:1185:d13 PAE L3 slot is shared
> (XEN) mm.c:1393:d13 Failure in alloc_l3_table: entry 4
> (XEN) mm.c:2070:d13 Error while validating mfn 79f0a (pfn 4c2) for type
> 60000000: caf=80000003 taf=60000001
> (XEN) mm.c:2700:d13 Error while pinning mfn 79f0a
> 
> I have not yet had any luck finding the cause of this. From what I
> can tell, the kernel's page tables are valid. The kernel initializes
> memory and performs many other initializations successfully before this
> point. Does anyone have any suggestions?

The fourth L2 page directory (that is, the one pointed at by the fourth and
final L3 pgdir entry) must not be shared by multiple L3 page directories.
You need one such L2 pogdir for every L3 pgdir. Many OSes would normally
share these, and will need modifying to not do so.

 -- Keir

Re: Question about MMUEXT_PIN_L3_TABLE

[W. Michael Petullo]

>> (XEN) mm.c:1185:d13 PAE L3 slot is shared
>> (XEN) mm.c:1393:d13 Failure in alloc_l3_table: entry 4
>> (XEN) mm.c:2070:d13 Error while validating mfn 79f0a (pfn 4c2) for type
>> 60000000: caf=80000003 taf=60000001
>> (XEN) mm.c:2700:d13 Error while pinning mfn 79f0a
>> 
>> I have not yet had any luck finding the cause of this. From what I
>> can tell, the kernel's page tables are valid. The kernel initializes
>> memory and performs many other initializations successfully before this
>> point. Does anyone have any suggestions?
 
> The fourth L2 page directory (that is, the one pointed at by the fourth and
> final L3 pgdir entry) must not be shared by multiple L3 page directories.
> You need one such L2 pogdir for every L3 pgdir. Many OSes would normally
> share these, and will need modifying to not do so.

As I understand this, I must create a new L2 page directory for my process
and copy the entries in the corresponding kernel PD into it. Is this
correct? Could you provide some insigt into why this is required? Do
I need to pin the L2 page directory before I pin the L3 page directory
pointer table?

-- 
Mike

:wq

Re: Question about MMUEXT_PIN_L3_TABLE

[Keir Fraser]

On 09/07/2010 12:23, "W. Michael Petullo" <mike@flyn.org> wrote:

>> The fourth L2 page directory (that is, the one pointed at by the fourth and
>> final L3 pgdir entry) must not be shared by multiple L3 page directories.
>> You need one such L2 pogdir for every L3 pgdir. Many OSes would normally
>> share these, and will need modifying to not do so.
> 
> As I understand this, I must create a new L2 page directory for my process
> and copy the entries in the corresponding kernel PD into it. Is this
> correct?

Yes.

>Could you provide some insigt into why this is required?

Xen owns private entries at the end of the fourth L2 page directory. Four of
these private entries map the contents of the L3 page directory entries
(yes, really) and obviously this arrangement only allows that fourth L2 page
directory to be mapped by one L3 page directory. Hence the unusual
restriction.

> Do
> I need to pin the L2 page directory before I pin the L3 page directory
> pointer table?

No. It gets implicitly pinned by the fact that it is mapped by a pinned L3.

 -- Keir