[PATCH] x86/PAE partially linear page table support

[PATCH] x86/PAE partially linear page table support

[Jan Beulich]

While full linear page table support makes little sense (and would be more
complicated to implement), partial linear page table support is almost identical
to that in non-PAE, and is used (at least) by NetWare.

Of course, the question here is whether this shouldn't be generalized - all
validated page tables (except for top level PAE) could be allowed to be installed
at lower levels using the same scheme.

Signed-off-by: Jan Beulich <jbeulich@novell.com>

Index: 2007-02-07/xen/arch/x86/mm.c
===================================================================
--- 2007-02-07.orig/xen/arch/x86/mm.c	2007-02-07 16:08:05.000000000 +0100
+++ 2007-02-07/xen/arch/x86/mm.c	2007-02-07 16:31:33.000000000 +0100
@@ -547,7 +547,7 @@ get_linear_pagetable(
             return 0;
 
         /*
-         * Make sure that the mapped frame is an already-validated L2 table. 
+         * Make sure that the mapped frame is an already-validated root table.
          * If so, atomically increment the count (checking for overflow).
          */
         page = mfn_to_page(pfn);
@@ -569,6 +569,51 @@ get_linear_pagetable(
 }
 #endif /* !CONFIG_X86_PAE */
 
+#if defined(CONFIG_X86_PAE) || defined(CONFIG_X86_64)
+static int
+get_l2_linear_pagetable(
+    l2_pgentry_t l2e, unsigned long l2e_pfn, struct domain *d)
+{
+    unsigned long pfn;
+
+    if ( (l2e_get_flags(l2e) & _PAGE_RW) )
+    {
+        MEM_LOG("Attempt to create linear p.t. with write perms");
+        return 0;
+    }
+
+    if ( (pfn = l2e_get_pfn(l2e)) != l2e_pfn )
+    {
+        unsigned long x, y;
+        struct page_info *page;
+
+        /* Make sure the mapped frame belongs to the correct domain. */
+        if ( unlikely(!get_page_from_pagenr(pfn, d)) )
+            return 0;
+
+        /*
+         * Make sure that the mapped frame is an already-validated L2 table.
+         * If so, atomically increment the count (checking for overflow).
+         */
+        page = mfn_to_page(pfn);
+        y = page->u.inuse.type_info;
+        do {
+            x = y;
+            if ( unlikely((x & PGT_count_mask) == PGT_count_mask) ||
+                 unlikely((x & (PGT_type_mask|PGT_validated)) !=
+                          (PGT_l2_page_table|PGT_validated)) )
+            {
+                put_page(page);
+                return 0;
+            }
+        }
+        while ( (y = cmpxchg(&page->u.inuse.type_info, x, x + 1)) != x );
+    }
+
+    return 1;
+}
+#endif /* !CONFIG_X86_PAE */
+
 int
 get_page_from_l1e(
     l1_pgentry_t l1e, struct domain *d)
@@ -645,10 +690,16 @@ get_page_from_l2e(
     }
 
     rc = get_page_and_type_from_pagenr(l2e_get_pfn(l2e), PGT_l1_page_table, d);
-#if CONFIG_PAGING_LEVELS == 2
     if ( unlikely(!rc) )
+    {
+#if CONFIG_PAGING_LEVELS == 2
         rc = get_linear_pagetable(l2e, pfn, d);
+#else
+        if ( (CONFIG_PAGING_LEVELS == 3 || IS_COMPAT(d)) )
+            rc = get_l2_linear_pagetable(l2e, pfn, d);
 #endif
+    }
+
     return rc;
 }

Re: [PATCH] x86/PAE partially linear page table support

[Keir Fraser]

How do you ensure that circular references are correctly cleaned up at
domain destruction time? (i.e., A references B and B references A).

We have special destructore code to clean up all root pagetable references,
but the code isn't there for lower levels.

 -- Keir

On 15/2/07 09:41, "Jan Beulich" <jbeulich@novell.com> wrote:

> While full linear page table support makes little sense (and would be more
> complicated to implement), partial linear page table support is almost
> identical
> to that in non-PAE, and is used (at least) by NetWare.
> 
> Of course, the question here is whether this shouldn't be generalized - all
> validated page tables (except for top level PAE) could be allowed to be
> installed
> at lower levels using the same scheme.
> 
> Signed-off-by: Jan Beulich <jbeulich@novell.com>

Re: [PATCH] x86/PAE partially linear page table support

[Jan Beulich]

>>> Keir Fraser <keir@xensource.com> 15.02.07 11:25 >>>
>
>How do you ensure that circular references are correctly cleaned up at
>domain destruction time? (i.e., A references B and B references A).
>
>We have special destructore code to clean up all root pagetable references,
>but the code isn't there for lower levels.

put_page_from_l?e() are all symmetrical, i.e. should take care of such
references at any level. I don't think I know what other destructor code
is involved here.

Jan

Re: [PATCH] x86/PAE partially linear page table support

[Keir Fraser]

On 15/2/07 11:41, "Jan Beulich" <jbeulich@novell.com> wrote:

>> How do you ensure that circular references are correctly cleaned up at
>> domain destruction time? (i.e., A references B and B references A).
>> 
>> We have special destructore code to clean up all root pagetable references,
>> but the code isn't there for lower levels.
> 
> put_page_from_l?e() are all symmetrical, i.e. should take care of such
> references at any level. I don't think I know what other destructor code
> is involved here.

That's not what I mean. Say we have two L2 tables, A and B. A[0] references
B and B[0] references A. Even when all external references to A and B have
gone away (i.e., there are no references from L3 tables) the type counts of
A and B will both be non-zero because of the circular reference. This will
prevent the pages from being freed and, ultimately, can lead to the domain
never being destroyed.

This case needs to be handled. See the code just below the largest comment
in arch/x86/domain.c:relinquish_memory() to see how it's handled for
top-level pagetables. Unfortunately it doesn't quite trivially generalise.

 -- Keir

Re: [PATCH] x86/PAE partially linear page table support

[Jan Beulich]

>That's not what I mean. Say we have two L2 tables, A and B. A[0] references
>B and B[0] references A. Even when all external references to A and B have
>gone away (i.e., there are no references from L3 tables) the type counts of
>A and B will both be non-zero because of the circular reference. This will
>prevent the pages from being freed and, ultimately, can lead to the domain
>never being destroyed.
>
>This case needs to be handled. See the code just below the largest comment
>in arch/x86/domain.c:relinquish_memory() to see how it's handled for
>top-level pagetables. Unfortunately it doesn't quite trivially generalise.

Hmm, it would seem to me that while it may not easily generalize, it at least
properly extends to the case the patch tries to address, by exchanging
PGT_base_page_table against PGT_l2_page_table. Really, in current code
this can never happen for PGT_base_page_table in PAE (or compat mode
guests), as such references aren't being established anywhere, and hence
the code should be dead for such guests.

But even for the general case - wouldn't it be possible to call
relinquish_memory() CONFIG_PAGING_LEVELS-1 times, each time lowering
the level of page tables that can be forced invalid by one?

Jan

Re: [PATCH] x86/PAE partially linear page table support

[Keir Fraser]

On 15/2/07 13:08, "Jan Beulich" <jbeulich@novell.com> wrote:

> Hmm, it would seem to me that while it may not easily generalize, it at least
> properly extends to the case the patch tries to address, by exchanging
> PGT_base_page_table against PGT_l2_page_table. Really, in current code
> this can never happen for PGT_base_page_table in PAE (or compat mode
> guests), as such references aren't being established anywhere, and hence
> the code should be dead for such guests.

You can't zap an L2 when there could be L3s later in the list that still
reference that L2.

> But even for the general case - wouldn't it be possible to call
> relinquish_memory() CONFIG_PAGING_LEVELS-1 times, each time lowering
> the level of page tables that can be forced invalid by one?

Yeah, this would work. Seems acceptable.

 -- Keir

Re: [PATCH] x86/PAE partially linear page table support

[Jan Beulich]

>>> Keir Fraser <keir@xensource.com> 15.02.07 14:24 >>>
>On 15/2/07 13:08, "Jan Beulich" <jbeulich@novell.com> wrote:
>> But even for the general case - wouldn't it be possible to call
>> relinquish_memory() CONFIG_PAGING_LEVELS-1 times, each time lowering
>> the level of page tables that can be forced invalid by one?
>
>Yeah, this would work. Seems acceptable.

Okay, so here's a second try:

While full linear page table support makes little sense (and would be more
complicated to implement), partial linear page table support is almost identical
to that in non-PAE, and is used (at least) by NetWare.

Of course, the question here is whether this shouldn't be generalized - all
validated page tables (except for top level PAE) could be allowed to be installed
at lower levels using the same scheme.

Signed-off-by: Jan Beulich <jbeulich@novell.com>

Index: 2007-02-07/xen/arch/x86/domain.c
===================================================================
--- 2007-02-07.orig/xen/arch/x86/domain.c	2007-02-07 16:27:53.000000000 +0100
+++ 2007-02-07/xen/arch/x86/domain.c	2007-02-15 15:23:47.000000000 +0100
@@ -1333,7 +1333,8 @@ int hypercall_xlat_continuation(unsigned
 }
 #endif
 
-static void relinquish_memory(struct domain *d, struct list_head *list)
+static void relinquish_memory(struct domain *d, struct list_head *list,
+                              unsigned long type)
 {
     struct list_head *ent;
     struct page_info  *page;
@@ -1362,23 +1363,24 @@ static void relinquish_memory(struct dom
             put_page(page);
 
         /*
-         * Forcibly invalidate base page tables at this point to break circular
-         * 'linear page table' references. This is okay because MMU structures
-         * are not shared across domains and this domain is now dead. Thus base
-         * tables are not in use so a non-zero count means circular reference.
+         * Forcibly invalidate top-most, still valid page tables at this point
+         * to break circular 'linear page table' references. This is okay
+         * because MMU structures are not shared across domains and this domain
+         * is now dead. Thus top-most valid tables are not in use so a non-zero
+         * count means circular reference.
          */
         y = page->u.inuse.type_info;
         for ( ; ; )
         {
             x = y;
             if ( likely((x & (PGT_type_mask|PGT_validated)) !=
-                        (PGT_base_page_table|PGT_validated)) )
+                        (type|PGT_validated)) )
                 break;
 
             y = cmpxchg(&page->u.inuse.type_info, x, x & ~PGT_validated);
             if ( likely(y == x) )
             {
-                free_page_type(page, PGT_base_page_table);
+                free_page_type(page, type);
                 break;
             }
         }
@@ -1476,8 +1478,16 @@ void domain_relinquish_resources(struct 
         destroy_gdt(v);
 
     /* Relinquish every page of memory. */
-    relinquish_memory(d, &d->xenpage_list);
-    relinquish_memory(d, &d->page_list);
+#if CONFIG_PAGING_LEVELS >= 4
+    relinquish_memory(d, &d->xenpage_list, PGT_l4_page_table);
+    relinquish_memory(d, &d->page_list, PGT_l4_page_table);
+#endif
+#if CONFIG_PAGING_LEVELS >= 3
+    relinquish_memory(d, &d->xenpage_list, PGT_l3_page_table);
+    relinquish_memory(d, &d->page_list, PGT_l3_page_table);
+#endif
+    relinquish_memory(d, &d->xenpage_list, PGT_l2_page_table);
+    relinquish_memory(d, &d->page_list, PGT_l2_page_table);
 
     /* Free page used by xen oprofile buffer */
     free_xenoprof_pages(d);
Index: 2007-02-07/xen/arch/x86/mm.c
===================================================================
--- 2007-02-07.orig/xen/arch/x86/mm.c	2007-02-07 16:08:05.000000000 +0100
+++ 2007-02-07/xen/arch/x86/mm.c	2007-02-07 16:31:33.000000000 +0100
@@ -547,7 +547,7 @@ get_linear_pagetable(
             return 0;
 
         /*
-         * Make sure that the mapped frame is an already-validated L2 table. 
+         * Make sure that the mapped frame is an already-validated root table.
          * If so, atomically increment the count (checking for overflow).
          */
         page = mfn_to_page(pfn);
@@ -569,6 +569,51 @@ get_linear_pagetable(
 }
 #endif /* !CONFIG_X86_PAE */
 
+#if defined(CONFIG_X86_PAE) || defined(CONFIG_X86_64)
+static int
+get_l2_linear_pagetable(
+    l2_pgentry_t l2e, unsigned long l2e_pfn, struct domain *d)
+{
+    unsigned long pfn;
+
+    if ( (l2e_get_flags(l2e) & _PAGE_RW) )
+    {
+        MEM_LOG("Attempt to create linear p.t. with write perms");
+        return 0;
+    }
+
+    if ( (pfn = l2e_get_pfn(l2e)) != l2e_pfn )
+    {
+        unsigned long x, y;
+        struct page_info *page;
+
+        /* Make sure the mapped frame belongs to the correct domain. */
+        if ( unlikely(!get_page_from_pagenr(pfn, d)) )
+            return 0;
+
+        /*
+         * Make sure that the mapped frame is an already-validated L2 table.
+         * If so, atomically increment the count (checking for overflow).
+         */
+        page = mfn_to_page(pfn);
+        y = page->u.inuse.type_info;
+        do {
+            x = y;
+            if ( unlikely((x & PGT_count_mask) == PGT_count_mask) ||
+                 unlikely((x & (PGT_type_mask|PGT_validated)) !=
+                          (PGT_l2_page_table|PGT_validated)) )
+            {
+                put_page(page);
+                return 0;
+            }
+        }
+        while ( (y = cmpxchg(&page->u.inuse.type_info, x, x + 1)) != x );
+    }
+
+    return 1;
+}
+#endif /* !CONFIG_X86_PAE */
+
 int
 get_page_from_l1e(
     l1_pgentry_t l1e, struct domain *d)
@@ -645,10 +690,16 @@ get_page_from_l2e(
     }
 
     rc = get_page_and_type_from_pagenr(l2e_get_pfn(l2e), PGT_l1_page_table, d);
-#if CONFIG_PAGING_LEVELS == 2
     if ( unlikely(!rc) )
+    {
+#if CONFIG_PAGING_LEVELS == 2
         rc = get_linear_pagetable(l2e, pfn, d);
+#else
+        if ( (CONFIG_PAGING_LEVELS == 3 || IS_COMPAT(d)) )
+            rc = get_l2_linear_pagetable(l2e, pfn, d);
 #endif
+    }
+
     return rc;
 }