* [PATCH 01/20] x86/shadow: Whitespace cleanup
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 02/20] x86/shadow: Rename hash_foreach() to hash_vcpu_foreach() Andrew Cooper
` (20 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 490 ++++++++++++------------
xen/arch/x86/mm/shadow/multi.c | 770 +++++++++++++++++++-------------------
xen/arch/x86/mm/shadow/multi.h | 36 +-
xen/arch/x86/mm/shadow/private.h | 106 +++---
xen/arch/x86/mm/shadow/types.h | 42 +--
5 files changed, 722 insertions(+), 722 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 3630ae0..502e0d8 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -5,7 +5,7 @@
* Parts of this code are Copyright (c) 2006 by XenSource Inc.
* Parts of this code are Copyright (c) 2006 by Michael A Fetterman
* Parts based on earlier work by Michael A Fetterman, Ian Pratt et al.
- *
+ *
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
@@ -252,7 +252,7 @@ hvm_emulate_write(enum x86_segment seg,
v, addr, p_data, bytes, sh_ctxt);
}
-static int
+static int
hvm_emulate_cmpxchg(enum x86_segment seg,
unsigned long offset,
void *p_old,
@@ -329,7 +329,7 @@ pv_emulate_write(enum x86_segment seg,
v, offset, p_data, bytes, sh_ctxt);
}
-static int
+static int
pv_emulate_cmpxchg(enum x86_segment seg,
unsigned long offset,
void *p_old,
@@ -409,9 +409,9 @@ const struct x86_emulate_ops *shadow_init_emulation(
return &hvm_shadow_emulator_ops;
}
-/* Update an initialized emulation context to prepare for the next
+/* Update an initialized emulation context to prepare for the next
* instruction */
-void shadow_continue_emulation(struct sh_emulate_ctxt *sh_ctxt,
+void shadow_continue_emulation(struct sh_emulate_ctxt *sh_ctxt,
struct cpu_user_regs *regs)
{
struct vcpu *v = current;
@@ -437,17 +437,17 @@ void shadow_continue_emulation(struct sh_emulate_ctxt *sh_ctxt,
}
}
}
-
+
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/**************************************************************************/
-/* Out-of-sync shadows. */
+/* Out-of-sync shadows. */
-/* From time to time, we let a shadowed pagetable page go out of sync
- * with its shadow: the guest is allowed to write directly to the page,
+/* From time to time, we let a shadowed pagetable page go out of sync
+ * with its shadow: the guest is allowed to write directly to the page,
* and those writes are not synchronously reflected in the shadow.
- * This lets us avoid many emulations if the guest is writing a lot to a
- * pagetable, but it relaxes a pretty important invariant in the shadow
+ * This lets us avoid many emulations if the guest is writing a lot to a
+ * pagetable, but it relaxes a pretty important invariant in the shadow
* pagetable design. Therefore, some rules:
*
* 1. Only L1 pagetables may go out of sync: any page that is shadowed
@@ -455,21 +455,21 @@ void shadow_continue_emulation(struct sh_emulate_ctxt *sh_ctxt,
* using linear shadow pagetables much less dangerous.
* That means that: (a) unsyncing code needs to check for higher-level
* shadows, and (b) promotion code needs to resync.
- *
+ *
* 2. All shadow operations on a guest page require the page to be brought
* back into sync before proceeding. This must be done under the
* paging lock so that the page is guaranteed to remain synced until
* the operation completes.
*
- * Exceptions to this rule: the pagefault and invlpg handlers may
- * update only one entry on an out-of-sync page without resyncing it.
+ * Exceptions to this rule: the pagefault and invlpg handlers may
+ * update only one entry on an out-of-sync page without resyncing it.
*
* 3. Operations on shadows that do not start from a guest page need to
* be aware that they may be handling an out-of-sync shadow.
*
- * 4. Operations that do not normally take the paging lock (fast-path
- * #PF handler, INVLPG) must fall back to a locking, syncing version
- * if they see an out-of-sync table.
+ * 4. Operations that do not normally take the paging lock (fast-path
+ * #PF handler, INVLPG) must fall back to a locking, syncing version
+ * if they see an out-of-sync table.
*
* 5. Operations corresponding to guest TLB flushes (MOV CR3, INVLPG)
* must explicitly resync all relevant pages or update their
@@ -488,26 +488,26 @@ void shadow_continue_emulation(struct sh_emulate_ctxt *sh_ctxt,
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES_FULL
-static void sh_oos_audit(struct domain *d)
+static void sh_oos_audit(struct domain *d)
{
int idx, expected_idx, expected_idx_alt;
struct page_info *pg;
struct vcpu *v;
-
- for_each_vcpu(d, v)
+
+ for_each_vcpu(d, v)
{
for ( idx = 0; idx < SHADOW_OOS_PAGES; idx++ )
{
mfn_t *oos = v->arch.paging.shadow.oos;
if ( !mfn_valid(oos[idx]) )
continue;
-
+
expected_idx = mfn_x(oos[idx]) % SHADOW_OOS_PAGES;
expected_idx_alt = ((expected_idx + 1) % SHADOW_OOS_PAGES);
if ( idx != expected_idx && idx != expected_idx_alt )
{
printk("%s: idx %d contains gmfn %lx, expected at %d or %d.\n",
- __func__, idx, mfn_x(oos[idx]),
+ __func__, idx, mfn_x(oos[idx]),
expected_idx, expected_idx_alt);
BUG();
}
@@ -536,21 +536,21 @@ static void sh_oos_audit(struct domain *d)
#endif
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES
-void oos_audit_hash_is_present(struct domain *d, mfn_t gmfn)
+void oos_audit_hash_is_present(struct domain *d, mfn_t gmfn)
{
int idx;
struct vcpu *v;
mfn_t *oos;
ASSERT(mfn_is_out_of_sync(gmfn));
-
- for_each_vcpu(d, v)
+
+ for_each_vcpu(d, v)
{
oos = v->arch.paging.shadow.oos;
idx = mfn_x(gmfn) % SHADOW_OOS_PAGES;
if ( mfn_x(oos[idx]) != mfn_x(gmfn) )
idx = (idx + 1) % SHADOW_OOS_PAGES;
-
+
if ( mfn_x(oos[idx]) == mfn_x(gmfn) )
return;
}
@@ -593,7 +593,7 @@ static inline int oos_fixup_flush_gmfn(struct vcpu *v, mfn_t gmfn,
if ( mfn_x(fixup->smfn[i]) != INVALID_MFN )
{
sh_remove_write_access_from_sl1p(v, gmfn,
- fixup->smfn[i],
+ fixup->smfn[i],
fixup->off[i]);
fixup->smfn[i] = _mfn(INVALID_MFN);
}
@@ -612,8 +612,8 @@ void oos_fixup_add(struct vcpu *v, mfn_t gmfn,
struct domain *d = v->domain;
perfc_incr(shadow_oos_fixup_add);
-
- for_each_vcpu(d, v)
+
+ for_each_vcpu(d, v)
{
oos = v->arch.paging.shadow.oos;
oos_fixup = v->arch.paging.shadow.oos_fixup;
@@ -638,7 +638,7 @@ void oos_fixup_add(struct vcpu *v, mfn_t gmfn,
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_OOS_FIXUP_EVICT);
/* Reuse this slot and remove current writable mapping. */
- sh_remove_write_access_from_sl1p(v, gmfn,
+ sh_remove_write_access_from_sl1p(v, gmfn,
oos_fixup[idx].smfn[next],
oos_fixup[idx].off[next]);
perfc_incr(shadow_oos_fixup_evict);
@@ -681,7 +681,7 @@ static int oos_remove_write_access(struct vcpu *v, mfn_t gmfn,
case -1:
/* An unfindable writeable typecount has appeared, probably via a
- * grant table entry: can't shoot the mapping, so try to unshadow
+ * grant table entry: can't shoot the mapping, so try to unshadow
* the page. If that doesn't work either, the guest is granting
* his pagetables and must be killed after all.
* This will flush the tlb, so we can return with no worries. */
@@ -715,7 +715,7 @@ static void _sh_resync(struct vcpu *v, mfn_t gmfn,
ASSERT(paging_locked_by_me(v->domain));
ASSERT(mfn_is_out_of_sync(gmfn));
/* Guest page must be shadowed *only* as L1 when out of sync. */
- ASSERT(!(mfn_to_page(gmfn)->shadow_flags & SHF_page_type_mask
+ ASSERT(!(mfn_to_page(gmfn)->shadow_flags & SHF_page_type_mask
& ~SHF_L1_ANY));
ASSERT(!sh_page_has_multiple_shadows(mfn_to_page(gmfn)));
@@ -751,14 +751,14 @@ static void oos_hash_add(struct vcpu *v, mfn_t gmfn)
mfn_t *oos_snapshot = v->arch.paging.shadow.oos_snapshot;
struct oos_fixup *oos_fixup = v->arch.paging.shadow.oos_fixup;
struct oos_fixup fixup = { .next = 0 };
-
+
for (i = 0; i < SHADOW_OOS_FIXUPS; i++ )
fixup.smfn[i] = _mfn(INVALID_MFN);
idx = mfn_x(gmfn) % SHADOW_OOS_PAGES;
oidx = idx;
- if ( mfn_valid(oos[idx])
+ if ( mfn_valid(oos[idx])
&& (mfn_x(oos[idx]) % SHADOW_OOS_PAGES) == idx )
{
/* Punt the current occupant into the next slot */
@@ -795,7 +795,7 @@ static void oos_hash_remove(struct vcpu *v, mfn_t gmfn)
SHADOW_PRINTK("%pv gmfn %lx\n", v, mfn_x(gmfn));
- for_each_vcpu(d, v)
+ for_each_vcpu(d, v)
{
oos = v->arch.paging.shadow.oos;
idx = mfn_x(gmfn) % SHADOW_OOS_PAGES;
@@ -818,8 +818,8 @@ mfn_t oos_snapshot_lookup(struct vcpu *v, mfn_t gmfn)
mfn_t *oos;
mfn_t *oos_snapshot;
struct domain *d = v->domain;
-
- for_each_vcpu(d, v)
+
+ for_each_vcpu(d, v)
{
oos = v->arch.paging.shadow.oos;
oos_snapshot = v->arch.paging.shadow.oos_snapshot;
@@ -846,7 +846,7 @@ void sh_resync(struct vcpu *v, mfn_t gmfn)
struct oos_fixup *oos_fixup;
struct domain *d = v->domain;
- for_each_vcpu(d, v)
+ for_each_vcpu(d, v)
{
oos = v->arch.paging.shadow.oos;
oos_fixup = v->arch.paging.shadow.oos_fixup;
@@ -854,7 +854,7 @@ void sh_resync(struct vcpu *v, mfn_t gmfn)
idx = mfn_x(gmfn) % SHADOW_OOS_PAGES;
if ( mfn_x(oos[idx]) != mfn_x(gmfn) )
idx = (idx + 1) % SHADOW_OOS_PAGES;
-
+
if ( mfn_x(oos[idx]) == mfn_x(gmfn) )
{
_sh_resync(v, gmfn, &oos_fixup[idx], oos_snapshot[idx]);
@@ -905,7 +905,7 @@ void sh_resync_all(struct vcpu *v, int skip, int this, int others)
goto resync_others;
/* First: resync all of this vcpu's oos pages */
- for ( idx = 0; idx < SHADOW_OOS_PAGES; idx++ )
+ for ( idx = 0; idx < SHADOW_OOS_PAGES; idx++ )
if ( mfn_valid(oos[idx]) )
{
/* Write-protect and sync contents */
@@ -920,14 +920,14 @@ void sh_resync_all(struct vcpu *v, int skip, int this, int others)
/* Second: make all *other* vcpus' oos pages safe. */
for_each_vcpu(v->domain, other)
{
- if ( v == other )
+ if ( v == other )
continue;
oos = other->arch.paging.shadow.oos;
oos_fixup = other->arch.paging.shadow.oos_fixup;
oos_snapshot = other->arch.paging.shadow.oos_snapshot;
- for ( idx = 0; idx < SHADOW_OOS_PAGES; idx++ )
+ for ( idx = 0; idx < SHADOW_OOS_PAGES; idx++ )
{
if ( !mfn_valid(oos[idx]) )
continue;
@@ -946,7 +946,7 @@ void sh_resync_all(struct vcpu *v, int skip, int this, int others)
_sh_resync(other, oos[idx], &oos_fixup[idx], oos_snapshot[idx]);
oos[idx] = _mfn(INVALID_MFN);
}
- }
+ }
}
}
@@ -955,19 +955,19 @@ void sh_resync_all(struct vcpu *v, int skip, int this, int others)
int sh_unsync(struct vcpu *v, mfn_t gmfn)
{
struct page_info *pg;
-
+
ASSERT(paging_locked_by_me(v->domain));
SHADOW_PRINTK("d=%d, v=%d, gmfn=%05lx\n",
v->domain->domain_id, v->vcpu_id, mfn_x(gmfn));
pg = mfn_to_page(gmfn);
-
+
/* Guest page must be shadowed *only* as L1 and *only* once when out
- * of sync. Also, get out now if it's already out of sync.
+ * of sync. Also, get out now if it's already out of sync.
* Also, can't safely unsync if some vcpus have paging disabled.*/
- if ( pg->shadow_flags &
- ((SHF_page_type_mask & ~SHF_L1_ANY) | SHF_out_of_sync)
+ if ( pg->shadow_flags &
+ ((SHF_page_type_mask & ~SHF_L1_ANY) | SHF_out_of_sync)
|| sh_page_has_multiple_shadows(pg)
|| is_pv_domain(v->domain)
|| !v->domain->arch.paging.shadow.oos_active )
@@ -995,9 +995,9 @@ void shadow_promote(struct vcpu *v, mfn_t gmfn, unsigned int type)
ASSERT(mfn_valid(gmfn));
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Is the page already shadowed and out of sync? */
- if ( page_is_out_of_sync(page) )
+ if ( page_is_out_of_sync(page) )
sh_resync(v, gmfn);
#endif
@@ -1026,13 +1026,13 @@ void shadow_demote(struct vcpu *v, mfn_t gmfn, u32 type)
if ( (page->shadow_flags & SHF_page_type_mask) == 0 )
{
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Was the page out of sync? */
- if ( page_is_out_of_sync(page) )
+ if ( page_is_out_of_sync(page) )
{
oos_hash_remove(v, gmfn);
}
-#endif
+#endif
clear_bit(_PGC_page_table, &page->count_info);
}
@@ -1050,11 +1050,11 @@ sh_validate_guest_entry(struct vcpu *v, mfn_t gmfn, void *entry, u32 size)
struct page_info *page = mfn_to_page(gmfn);
paging_mark_dirty(v->domain, mfn_x(gmfn));
-
+
// Determine which types of shadows are affected, and update each.
//
// Always validate L1s before L2s to prevent another cpu with a linear
- // mapping of this gmfn from seeing a walk that results from
+ // mapping of this gmfn from seeing a walk that results from
// using the new L2 value and the old L1 value. (It is OK for such a
// guest to see a walk that uses the old L2 value with the new L1 value,
// as hardware could behave this way if one level of the pagewalk occurs
@@ -1067,40 +1067,40 @@ sh_validate_guest_entry(struct vcpu *v, mfn_t gmfn, void *entry, u32 size)
if ( !(page->count_info & PGC_page_table) )
return 0; /* Not shadowed at all */
- if ( page->shadow_flags & SHF_L1_32 )
+ if ( page->shadow_flags & SHF_L1_32 )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl1e, 2)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L2_32 )
+ if ( page->shadow_flags & SHF_L2_32 )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl2e, 2)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L1_PAE )
+ if ( page->shadow_flags & SHF_L1_PAE )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl1e, 3)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L2_PAE )
+ if ( page->shadow_flags & SHF_L2_PAE )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl2e, 3)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L2H_PAE )
+ if ( page->shadow_flags & SHF_L2H_PAE )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl2he, 3)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L1_64 )
+ if ( page->shadow_flags & SHF_L1_64 )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl1e, 4)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L2_64 )
+ if ( page->shadow_flags & SHF_L2_64 )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl2e, 4)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L2H_64 )
+ if ( page->shadow_flags & SHF_L2H_64 )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl2he, 4)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L3_64 )
+ if ( page->shadow_flags & SHF_L3_64 )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl3e, 4)
(v, gmfn, entry, size);
- if ( page->shadow_flags & SHF_L4_64 )
+ if ( page->shadow_flags & SHF_L4_64 )
result |= SHADOW_INTERNAL_NAME(sh_map_and_validate_gl4e, 4)
(v, gmfn, entry, size);
- this_cpu(trace_shadow_path_flags) |= (result<<(TRCE_SFLAG_SET_CHANGED));
+ this_cpu(trace_shadow_path_flags) |= (result<<(TRCE_SFLAG_SET_CHANGED));
return result;
}
@@ -1121,12 +1121,12 @@ sh_validate_guest_pt_write(struct vcpu *v, mfn_t gmfn,
if ( rc & SHADOW_SET_FLUSH )
/* Need to flush TLBs to pick up shadow PT changes */
flush_tlb_mask(d->domain_dirty_cpumask);
- if ( rc & SHADOW_SET_ERROR )
+ if ( rc & SHADOW_SET_ERROR )
{
- /* This page is probably not a pagetable any more: tear it out of the
- * shadows, along with any tables that reference it.
- * Since the validate call above will have made a "safe" (i.e. zero)
- * shadow entry, we can let the domain live even if we can't fully
+ /* This page is probably not a pagetable any more: tear it out of the
+ * shadows, along with any tables that reference it.
+ * Since the validate call above will have made a "safe" (i.e. zero)
+ * shadow entry, we can let the domain live even if we can't fully
* unshadow the page. */
sh_remove_shadows(v, gmfn, 0, 0);
}
@@ -1134,7 +1134,7 @@ sh_validate_guest_pt_write(struct vcpu *v, mfn_t gmfn,
int shadow_write_guest_entry(struct vcpu *v, intpte_t *p,
intpte_t new, mfn_t gmfn)
-/* Write a new value into the guest pagetable, and update the shadows
+/* Write a new value into the guest pagetable, and update the shadows
* appropriately. Returns 0 if we page-faulted, 1 for success. */
{
int failed;
@@ -1148,7 +1148,7 @@ int shadow_write_guest_entry(struct vcpu *v, intpte_t *p,
int shadow_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p,
intpte_t *old, intpte_t new, mfn_t gmfn)
-/* Cmpxchg a new value into the guest pagetable, and update the shadows
+/* Cmpxchg a new value into the guest pagetable, and update the shadows
* appropriately. Returns 0 if we page-faulted, 1 if not.
* N.B. caller should check the value of "old" to see if the
* cmpxchg itself was successful. */
@@ -1166,7 +1166,7 @@ int shadow_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p,
/**************************************************************************/
-/* Memory management for shadow pages. */
+/* Memory management for shadow pages. */
/* Allocating shadow pages
* -----------------------
@@ -1180,12 +1180,12 @@ int shadow_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p,
* PAE/64-bit l2 tables (1GB va each). These multi-page shadows are
* not contiguous in memory; functions for handling offsets into them are
* defined in shadow/multi.c (shadow_l1_index() etc.)
- *
+ *
* This table shows the allocation behaviour of the different modes:
*
* Xen paging 64b 64b 64b
* Guest paging 32b pae 64b
- * PV or HVM HVM HVM *
+ * PV or HVM HVM HVM *
* Shadow paging pae pae 64b
*
* sl1 size 8k 4k 4k
@@ -1193,8 +1193,8 @@ int shadow_cmpxchg_guest_entry(struct vcpu *v, intpte_t *p,
* sl3 size - - 4k
* sl4 size - - 4k
*
- * In HVM guests, the p2m table is built out of shadow pages, and we provide
- * a function for the p2m management to steal pages, in max-order chunks, from
+ * In HVM guests, the p2m table is built out of shadow pages, and we provide
+ * a function for the p2m management to steal pages, in max-order chunks, from
* the free pool.
*/
@@ -1221,15 +1221,15 @@ const u8 sh_type_to_size[] = {
/* Figure out the least acceptable quantity of shadow memory.
* The minimum memory requirement for always being able to free up a
* chunk of memory is very small -- only three max-order chunks per
- * vcpu to hold the top level shadows and pages with Xen mappings in them.
+ * vcpu to hold the top level shadows and pages with Xen mappings in them.
*
* But for a guest to be guaranteed to successfully execute a single
* instruction, we must be able to map a large number (about thirty) VAs
* at the same time, which means that to guarantee progress, we must
* allow for more than ninety allocated pages per vcpu. We round that
- * up to 128 pages, or half a megabyte per vcpu, and add 1 more vcpu's
+ * up to 128 pages, or half a megabyte per vcpu, and add 1 more vcpu's
* worth to make sure we never return zero. */
-static unsigned int shadow_min_acceptable_pages(struct domain *d)
+static unsigned int shadow_min_acceptable_pages(struct domain *d)
{
u32 vcpu_count = 1;
struct vcpu *v;
@@ -1238,7 +1238,7 @@ static unsigned int shadow_min_acceptable_pages(struct domain *d)
vcpu_count++;
return (vcpu_count * 128);
-}
+}
/* Dispatcher function: call the per-mode function that will unhook the
* non-Xen mappings in this top-level shadow mfn. With user_only == 1,
@@ -1290,7 +1290,7 @@ static void _shadow_prealloc(
int i;
if ( d->arch.paging.shadow.free_pages >= pages ) return;
-
+
v = current;
if ( v->domain != d )
v = d->vcpu[0];
@@ -1315,13 +1315,13 @@ static void _shadow_prealloc(
* mappings. */
perfc_incr(shadow_prealloc_2);
- for_each_vcpu(d, v2)
+ for_each_vcpu(d, v2)
for ( i = 0 ; i < 4 ; i++ )
{
if ( !pagetable_is_null(v2->arch.shadow_table[i]) )
{
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_PREALLOC_UNHOOK);
- shadow_unhook_mappings(v,
+ shadow_unhook_mappings(v,
pagetable_get_mfn(v2->arch.shadow_table[i]), 0);
/* See if that freed up enough space */
@@ -1332,7 +1332,7 @@ static void _shadow_prealloc(
}
}
}
-
+
/* Nothing more we can do: all remaining shadows are of pages that
* hold Xen mappings for some vcpu. This can never happen. */
SHADOW_ERROR("Can't pre-allocate %u shadow pages!\n"
@@ -1356,7 +1356,7 @@ void shadow_prealloc(struct domain *d, u32 type, unsigned int count)
/* Deliberately free all the memory we can: this will tear down all of
* this domain's shadows */
-static void shadow_blow_tables(struct domain *d)
+static void shadow_blow_tables(struct domain *d)
{
struct page_info *sp, *t;
struct vcpu *v = d->vcpu[0];
@@ -1371,12 +1371,12 @@ static void shadow_blow_tables(struct domain *d)
smfn = page_to_mfn(sp);
sh_unpin(v, smfn);
}
-
+
/* Second pass: unhook entries of in-use shadows */
- for_each_vcpu(d, v)
+ for_each_vcpu(d, v)
for ( i = 0 ; i < 4 ; i++ )
if ( !pagetable_is_null(v->arch.shadow_table[i]) )
- shadow_unhook_mappings(v,
+ shadow_unhook_mappings(v,
pagetable_get_mfn(v->arch.shadow_table[i]), 0);
/* Make sure everyone sees the unshadowings */
@@ -1441,9 +1441,9 @@ set_next_shadow(struct page_info *sp, struct page_info *next)
}
/* Allocate another shadow's worth of (contiguous, aligned) pages,
- * and fill in the type and backpointer fields of their page_infos.
+ * and fill in the type and backpointer fields of their page_infos.
* Never fails to allocate. */
-mfn_t shadow_alloc(struct domain *d,
+mfn_t shadow_alloc(struct domain *d,
u32 shadow_type,
unsigned long backpointer)
{
@@ -1485,10 +1485,10 @@ mfn_t shadow_alloc(struct domain *d,
INIT_PAGE_LIST_HEAD(&tmp_list);
/* Init page info fields and clear the pages */
- for ( i = 0; i < pages ; i++ )
+ for ( i = 0; i < pages ; i++ )
{
sp = page_list_remove_head(&d->arch.paging.shadow.freelist);
- /* Before we overwrite the old contents of this page,
+ /* Before we overwrite the old contents of this page,
* we need to be sure that no TLB holds a pointer to it. */
cpumask_copy(&mask, d->domain_dirty_cpumask);
tlbflush_filter(mask, sp->tlbflush_timestamp);
@@ -1512,7 +1512,7 @@ mfn_t shadow_alloc(struct domain *d,
set_next_shadow(sp, NULL);
perfc_incr(shadow_alloc_count);
}
- if ( shadow_type >= SH_type_min_shadow
+ if ( shadow_type >= SH_type_min_shadow
&& shadow_type <= SH_type_max_shadow )
sp->u.sh.head = 1;
@@ -1525,7 +1525,7 @@ mfn_t shadow_alloc(struct domain *d,
/* Return some shadow pages to the pool. */
void shadow_free(struct domain *d, mfn_t smfn)
{
- struct page_info *next = NULL, *sp = mfn_to_page(smfn);
+ struct page_info *next = NULL, *sp = mfn_to_page(smfn);
struct page_list_head *pin_list;
unsigned int pages;
u32 shadow_type;
@@ -1540,16 +1540,16 @@ void shadow_free(struct domain *d, mfn_t smfn)
pages = shadow_size(shadow_type);
pin_list = &d->arch.paging.shadow.pinned_shadows;
- for ( i = 0; i < pages; i++ )
+ for ( i = 0; i < pages; i++ )
{
#if SHADOW_OPTIMIZATIONS & (SHOPT_WRITABLE_HEURISTIC | SHOPT_FAST_EMULATION)
struct vcpu *v;
- for_each_vcpu(d, v)
+ for_each_vcpu(d, v)
{
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
/* No longer safe to look for a writeable mapping in this shadow */
- if ( v->arch.paging.shadow.last_writeable_pte_smfn
- == mfn_x(page_to_mfn(sp)) )
+ if ( v->arch.paging.shadow.last_writeable_pte_smfn
+ == mfn_x(page_to_mfn(sp)) )
v->arch.paging.shadow.last_writeable_pte_smfn = 0;
#endif
#if SHADOW_OPTIMIZATIONS & SHOPT_FAST_EMULATION
@@ -1562,7 +1562,7 @@ void shadow_free(struct domain *d, mfn_t smfn)
next = page_list_next(sp, pin_list);
/* Strip out the type: this is now a free shadow page */
sp->u.sh.type = sp->u.sh.head = 0;
- /* Remember the TLB timestamp so we will know whether to flush
+ /* Remember the TLB timestamp so we will know whether to flush
* TLBs when we reuse the page. Because the destructors leave the
* contents of the pages in place, we can delay TLB flushes until
* just before the allocator hands the page out again. */
@@ -1584,11 +1584,11 @@ shadow_alloc_p2m_page(struct domain *d)
{
struct page_info *pg;
- /* This is called both from the p2m code (which never holds the
+ /* This is called both from the p2m code (which never holds the
* paging lock) and the log-dirty code (which always does). */
paging_lock_recursive(d);
- if ( d->arch.paging.shadow.total_pages
+ if ( d->arch.paging.shadow.total_pages
< shadow_min_acceptable_pages(d) + 1 )
{
if ( !d->arch.paging.p2m_alloc_failed )
@@ -1630,9 +1630,9 @@ shadow_free_p2m_page(struct domain *d, struct page_info *pg)
}
pg->count_info &= ~PGC_count_mask;
pg->u.sh.type = SH_type_p2m_table; /* p2m code reuses type-info */
- page_set_owner(pg, NULL);
+ page_set_owner(pg, NULL);
- /* This is called both from the p2m code (which never holds the
+ /* This is called both from the p2m code (which never holds the
* paging lock) and the log-dirty code (which always does). */
paging_lock_recursive(d);
@@ -1647,7 +1647,7 @@ shadow_free_p2m_page(struct domain *d, struct page_info *pg)
* Input will be rounded up to at least shadow_min_acceptable_pages(),
* plus space for the p2m table.
* Returns 0 for success, non-zero for failure. */
-static unsigned int sh_set_allocation(struct domain *d,
+static unsigned int sh_set_allocation(struct domain *d,
unsigned int pages,
int *preempted)
{
@@ -1663,7 +1663,7 @@ static unsigned int sh_set_allocation(struct domain *d,
pages = 0;
else
pages -= d->arch.paging.shadow.p2m_pages;
-
+
/* Don't allocate less than the minimum acceptable, plus one page per
* megabyte of RAM (for the p2m table) */
lower_bound = shadow_min_acceptable_pages(d) + (d->tot_pages / 256);
@@ -1671,18 +1671,18 @@ static unsigned int sh_set_allocation(struct domain *d,
pages = lower_bound;
}
- SHADOW_PRINTK("current %i target %i\n",
+ SHADOW_PRINTK("current %i target %i\n",
d->arch.paging.shadow.total_pages, pages);
for ( ; ; )
{
- if ( d->arch.paging.shadow.total_pages < pages )
+ if ( d->arch.paging.shadow.total_pages < pages )
{
/* Need to allocate more memory from domheap */
sp = (struct page_info *)
alloc_domheap_page(NULL, MEMF_node(domain_to_node(d)));
- if ( sp == NULL )
- {
+ if ( sp == NULL )
+ {
SHADOW_PRINTK("failed to allocate shadow pages.\n");
return -ENOMEM;
}
@@ -1693,8 +1693,8 @@ static unsigned int sh_set_allocation(struct domain *d,
sp->u.sh.count = 0;
sp->tlbflush_timestamp = 0; /* Not in any TLB */
page_list_add_tail(sp, &d->arch.paging.shadow.freelist);
- }
- else if ( d->arch.paging.shadow.total_pages > pages )
+ }
+ else if ( d->arch.paging.shadow.total_pages > pages )
{
/* Need to return memory to domheap */
_shadow_prealloc(d, 1);
@@ -1734,7 +1734,7 @@ static unsigned int shadow_get_allocation(struct domain *d)
/**************************************************************************/
/* Hash table for storing the guest->shadow mappings.
- * The table itself is an array of pointers to shadows; the shadows are then
+ * The table itself is an array of pointers to shadows; the shadows are then
* threaded on a singly-linked list of shadows with the same hash value */
#define SHADOW_HASH_BUCKETS 251
@@ -1742,7 +1742,7 @@ static unsigned int shadow_get_allocation(struct domain *d)
/* Hash function that takes a gfn or mfn, plus another byte of type info */
typedef u32 key_t;
-static inline key_t sh_hash(unsigned long n, unsigned int t)
+static inline key_t sh_hash(unsigned long n, unsigned int t)
{
unsigned char *p = (unsigned char *)&n;
key_t k = t;
@@ -1801,7 +1801,7 @@ static void sh_hash_audit_bucket(struct domain *d, int bucket)
SHADOW_ERROR("MFN %#"PRI_mfn" shadowed (by %#"PRI_mfn")"
" and not OOS but has typecount %#lx\n",
__backpointer(sp),
- mfn_x(page_to_mfn(sp)),
+ mfn_x(page_to_mfn(sp)),
gpg->u.inuse.type_info);
BUG();
}
@@ -1809,7 +1809,7 @@ static void sh_hash_audit_bucket(struct domain *d, int bucket)
}
else /* Not an l1 */
#endif
- if ( (gpg->u.inuse.type_info & PGT_type_mask) == PGT_writable_page
+ if ( (gpg->u.inuse.type_info & PGT_type_mask) == PGT_writable_page
&& (gpg->u.inuse.type_info & PGT_count_mask) != 0 )
{
SHADOW_ERROR("MFN %#"PRI_mfn" shadowed (by %#"PRI_mfn")"
@@ -1839,7 +1839,7 @@ static void sh_hash_audit(struct domain *d)
if ( !(SHADOW_AUDIT_ENABLE) )
return;
- for ( i = 0; i < SHADOW_HASH_BUCKETS; i++ )
+ for ( i = 0; i < SHADOW_HASH_BUCKETS; i++ )
{
sh_hash_audit_bucket(d, i);
}
@@ -1849,7 +1849,7 @@ static void sh_hash_audit(struct domain *d)
#define sh_hash_audit(_d) do {} while(0)
#endif /* Hashtable bucket audit */
-/* Allocate and initialise the table itself.
+/* Allocate and initialise the table itself.
* Returns 0 for success, 1 for error. */
static int shadow_hash_alloc(struct domain *d)
{
@@ -1906,11 +1906,11 @@ mfn_t shadow_hash_lookup(struct vcpu *v, unsigned long n, unsigned int t)
if ( unlikely(d->arch.paging.shadow.hash_walking != 0) )
/* Can't reorder: someone is walking the hash chains */
return page_to_mfn(sp);
- else
+ else
{
ASSERT(prev);
/* Delete sp from the list */
- prev->next_shadow = sp->next_shadow;
+ prev->next_shadow = sp->next_shadow;
/* Re-insert it at the head of the list */
set_next_shadow(sp, d->arch.paging.shadow.hash_table[key]);
d->arch.paging.shadow.hash_table[key] = sp;
@@ -1930,14 +1930,14 @@ mfn_t shadow_hash_lookup(struct vcpu *v, unsigned long n, unsigned int t)
return _mfn(INVALID_MFN);
}
-void shadow_hash_insert(struct vcpu *v, unsigned long n, unsigned int t,
+void shadow_hash_insert(struct vcpu *v, unsigned long n, unsigned int t,
mfn_t smfn)
/* Put a mapping (n,t)->smfn into the hash table */
{
struct domain *d = v->domain;
struct page_info *sp;
key_t key;
-
+
ASSERT(paging_locked_by_me(d));
ASSERT(d->arch.paging.shadow.hash_table);
ASSERT(t);
@@ -1947,16 +1947,16 @@ void shadow_hash_insert(struct vcpu *v, unsigned long n, unsigned int t,
perfc_incr(shadow_hash_inserts);
key = sh_hash(n, t);
sh_hash_audit_bucket(d, key);
-
+
/* Insert this shadow at the top of the bucket */
sp = mfn_to_page(smfn);
set_next_shadow(sp, d->arch.paging.shadow.hash_table[key]);
d->arch.paging.shadow.hash_table[key] = sp;
-
+
sh_hash_audit_bucket(d, key);
}
-void shadow_hash_delete(struct vcpu *v, unsigned long n, unsigned int t,
+void shadow_hash_delete(struct vcpu *v, unsigned long n, unsigned int t,
mfn_t smfn)
/* Excise the mapping (n,t)->smfn from the hash table */
{
@@ -1973,12 +1973,12 @@ void shadow_hash_delete(struct vcpu *v, unsigned long n, unsigned int t,
perfc_incr(shadow_hash_deletes);
key = sh_hash(n, t);
sh_hash_audit_bucket(d, key);
-
+
sp = mfn_to_page(smfn);
- if ( d->arch.paging.shadow.hash_table[key] == sp )
+ if ( d->arch.paging.shadow.hash_table[key] == sp )
/* Easy case: we're deleting the head item. */
d->arch.paging.shadow.hash_table[key] = next_shadow(sp);
- else
+ else
{
/* Need to search for the one we want */
x = d->arch.paging.shadow.hash_table[key];
@@ -2001,17 +2001,17 @@ void shadow_hash_delete(struct vcpu *v, unsigned long n, unsigned int t,
typedef int (*hash_callback_t)(struct vcpu *v, mfn_t smfn, mfn_t other_mfn);
-static void hash_foreach(struct vcpu *v,
- unsigned int callback_mask,
+static void hash_foreach(struct vcpu *v,
+ unsigned int callback_mask,
const hash_callback_t callbacks[],
mfn_t callback_mfn)
-/* Walk the hash table looking at the types of the entries and
- * calling the appropriate callback function for each entry.
+/* Walk the hash table looking at the types of the entries and
+ * calling the appropriate callback function for each entry.
* The mask determines which shadow types we call back for, and the array
* of callbacks tells us which function to call.
- * Any callback may return non-zero to let us skip the rest of the scan.
+ * Any callback may return non-zero to let us skip the rest of the scan.
*
- * WARNING: Callbacks MUST NOT add or remove hash entries unless they
+ * WARNING: Callbacks MUST NOT add or remove hash entries unless they
* then return non-zero to terminate the scan. */
{
int i, done = 0;
@@ -2028,7 +2028,7 @@ static void hash_foreach(struct vcpu *v,
ASSERT(d->arch.paging.shadow.hash_walking == 0);
d->arch.paging.shadow.hash_walking = 1;
- for ( i = 0; i < SHADOW_HASH_BUCKETS; i++ )
+ for ( i = 0; i < SHADOW_HASH_BUCKETS; i++ )
{
/* WARNING: This is not safe against changes to the hash table.
* The callback *must* return non-zero if it has inserted or
@@ -2044,15 +2044,15 @@ static void hash_foreach(struct vcpu *v,
if ( done ) break;
}
}
- if ( done ) break;
+ if ( done ) break;
}
- d->arch.paging.shadow.hash_walking = 0;
+ d->arch.paging.shadow.hash_walking = 0;
}
/**************************************************************************/
/* Destroy a shadow page: simple dispatcher to call the per-type destructor
- * which will decrement refcounts appropriately and return memory to the
+ * which will decrement refcounts appropriately and return memory to the
* free pool. */
void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
@@ -2065,13 +2065,13 @@ void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
/* Double-check, if we can, that the shadowed page belongs to this
* domain, (by following the back-pointer). */
- ASSERT(t == SH_type_fl1_32_shadow ||
- t == SH_type_fl1_pae_shadow ||
- t == SH_type_fl1_64_shadow ||
- t == SH_type_monitor_table ||
+ ASSERT(t == SH_type_fl1_32_shadow ||
+ t == SH_type_fl1_pae_shadow ||
+ t == SH_type_fl1_64_shadow ||
+ t == SH_type_monitor_table ||
(is_pv_32on64_vcpu(v) && t == SH_type_l4_64_shadow) ||
(page_get_owner(mfn_to_page(backpointer(sp)))
- == v->domain));
+ == v->domain));
/* The down-shifts here are so that the switch statement is on nice
* small numbers that the compiler will enjoy */
@@ -2115,7 +2115,7 @@ void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
SHADOW_ERROR("tried to destroy shadow of bad type %08lx\n",
(unsigned long)t);
BUG();
- }
+ }
}
static inline void trace_shadow_wrmap_bf(mfn_t gmfn)
@@ -2129,13 +2129,13 @@ static inline void trace_shadow_wrmap_bf(mfn_t gmfn)
}
/**************************************************************************/
-/* Remove all writeable mappings of a guest frame from the shadow tables
- * Returns non-zero if we need to flush TLBs.
+/* Remove all writeable mappings of a guest frame from the shadow tables
+ * Returns non-zero if we need to flush TLBs.
* level and fault_addr desribe how we found this to be a pagetable;
* level==0 means we have some other reason for revoking write access.
* If level==0 we are allowed to fail, returning -1. */
-int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
+int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
unsigned int level,
unsigned long fault_addr)
{
@@ -2180,7 +2180,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
/* Early exit if it's already a pagetable, or otherwise not writeable */
if ( (sh_mfn_is_a_page_table(gmfn)
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Unless they've been allowed to go out of sync with their shadows */
&& !mfn_oos_may_write(gmfn)
#endif
@@ -2192,11 +2192,11 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
perfc_incr(shadow_writeable);
- /* If this isn't a "normal" writeable page, the domain is trying to
+ /* If this isn't a "normal" writeable page, the domain is trying to
* put pagetables in special memory of some kind. We can't allow that. */
if ( (pg->u.inuse.type_info & PGT_type_mask) != PGT_writable_page )
{
- SHADOW_ERROR("can't remove write access to mfn %lx, type_info is %"
+ SHADOW_ERROR("can't remove write access to mfn %lx, type_info is %"
PRtype_info "\n",
mfn_x(gmfn), mfn_to_page(gmfn)->u.inuse.type_info);
domain_crash(v->domain);
@@ -2219,7 +2219,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
return 1; \
} \
} while (0)
-
+
if ( v->arch.paging.mode->guest_levels == 2 )
{
if ( level == 1 )
@@ -2227,27 +2227,27 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
GUESS(0xC0000000UL + (fault_addr >> 10), 1);
/* Linux lowmem: first 896MB is mapped 1-to-1 above 0xC0000000 */
- if ((gfn = mfn_to_gfn(v->domain, gmfn)) < 0x38000 )
+ if ((gfn = mfn_to_gfn(v->domain, gmfn)) < 0x38000 )
GUESS(0xC0000000UL + (gfn << PAGE_SHIFT), 4);
/* FreeBSD: Linear map at 0xBFC00000 */
if ( level == 1 )
- GUESS(0xBFC00000UL
+ GUESS(0xBFC00000UL
+ ((fault_addr & VADDR_MASK) >> 10), 6);
}
else if ( v->arch.paging.mode->guest_levels == 3 )
{
/* 32bit PAE w2k3: linear map at 0xC0000000 */
- switch ( level )
+ switch ( level )
{
case 1: GUESS(0xC0000000UL + (fault_addr >> 9), 2); break;
case 2: GUESS(0xC0600000UL + (fault_addr >> 18), 2); break;
}
/* Linux lowmem: first 896MB is mapped 1-to-1 above 0xC0000000 */
- if ((gfn = mfn_to_gfn(v->domain, gmfn)) < 0x38000 )
+ if ((gfn = mfn_to_gfn(v->domain, gmfn)) < 0x38000 )
GUESS(0xC0000000UL + (gfn << PAGE_SHIFT), 4);
-
+
/* FreeBSD PAE: Linear map at 0xBF800000 */
switch ( level )
{
@@ -2260,20 +2260,20 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
else if ( v->arch.paging.mode->guest_levels == 4 )
{
/* 64bit w2k3: linear map at 0xfffff68000000000 */
- switch ( level )
+ switch ( level )
{
- case 1: GUESS(0xfffff68000000000UL
+ case 1: GUESS(0xfffff68000000000UL
+ ((fault_addr & VADDR_MASK) >> 9), 3); break;
case 2: GUESS(0xfffff6fb40000000UL
+ ((fault_addr & VADDR_MASK) >> 18), 3); break;
- case 3: GUESS(0xfffff6fb7da00000UL
+ case 3: GUESS(0xfffff6fb7da00000UL
+ ((fault_addr & VADDR_MASK) >> 27), 3); break;
}
/* 64bit Linux direct map at 0xffff880000000000; older kernels
* had it at 0xffff810000000000, and older kernels yet had it
* at 0x0000010000000000UL */
- gfn = mfn_to_gfn(v->domain, gmfn);
+ gfn = mfn_to_gfn(v->domain, gmfn);
GUESS(0xffff880000000000UL + (gfn << PAGE_SHIFT), 4);
GUESS(0xffff810000000000UL + (gfn << PAGE_SHIFT), 4);
GUESS(0x0000010000000000UL + (gfn << PAGE_SHIFT), 4);
@@ -2283,7 +2283,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
* kpm_vbase; 0xfffffe0000000000UL
*/
GUESS(0xfffffe0000000000UL + (gfn << PAGE_SHIFT), 4);
-
+
/* FreeBSD 64bit: linear map 0xffff800000000000 */
switch ( level )
{
@@ -2316,7 +2316,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
mfn_t last_smfn = _mfn(v->arch.paging.shadow.last_writeable_pte_smfn);
int shtype = mfn_to_page(last_smfn)->u.sh.type;
- if ( callbacks[shtype] )
+ if ( callbacks[shtype] )
callbacks[shtype](v, last_smfn, gmfn);
if ( (pg->u.inuse.type_info & PGT_count_mask) != old_count )
@@ -2327,7 +2327,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
return 1;
#endif /* SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC */
-
+
/* Brute-force search of all the shadows, by walking the hash */
trace_shadow_wrmap_bf(gmfn);
if ( level == 0 )
@@ -2348,20 +2348,20 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
(mfn_to_page(gmfn)->u.inuse.type_info&PGT_count_mask));
domain_crash(v->domain);
}
-
+
/* We killed at least one writeable mapping, so must flush TLBs. */
return 1;
}
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
int sh_remove_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
mfn_t smfn, unsigned long off)
{
struct page_info *sp = mfn_to_page(smfn);
-
+
ASSERT(mfn_valid(smfn));
ASSERT(mfn_valid(gmfn));
-
+
if ( sp->u.sh.type == SH_type_l1_32_shadow
|| sp->u.sh.type == SH_type_fl1_32_shadow )
{
@@ -2379,7 +2379,7 @@ int sh_remove_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
return 0;
}
-#endif
+#endif
/**************************************************************************/
/* Remove all mappings of a guest frame from the shadow tables.
@@ -2427,9 +2427,9 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
* can be called via put_page_type when we clear a shadow l1e).*/
paging_lock_recursive(v->domain);
- /* XXX TODO:
+ /* XXX TODO:
* Heuristics for finding the (probably) single mapping of this gmfn */
-
+
/* Brute-force search of all the shadows, by walking the hash */
perfc_incr(shadow_mappings_bf);
hash_foreach(v, callback_mask, callbacks, gmfn);
@@ -2437,8 +2437,8 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
/* If that didn't catch the mapping, something is very wrong */
if ( !sh_check_page_has_no_refs(page) )
{
- /* Don't complain if we're in HVM and there are some extra mappings:
- * The qemu helper process has an untyped mapping of this dom's RAM
+ /* Don't complain if we're in HVM and there are some extra mappings:
+ * The qemu helper process has an untyped mapping of this dom's RAM
* and the HVM restore program takes another.
* Also allow one typed refcount for xenheap pages, to match
* share_xen_page_with_guest(). */
@@ -2448,7 +2448,7 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
== !!is_xen_heap_page(page))) )
{
SHADOW_ERROR("can't find all mappings of mfn %lx: "
- "c=%08lx t=%08lx\n", mfn_x(gmfn),
+ "c=%08lx t=%08lx\n", mfn_x(gmfn),
page->count_info, page->u.inuse.type_info);
}
}
@@ -2475,7 +2475,7 @@ static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
ASSERT(sp->u.sh.type > 0);
ASSERT(sp->u.sh.type < SH_type_max_shadow);
ASSERT(sh_type_has_up_pointer(v, sp->u.sh.type));
-
+
if (sp->up == 0) return 0;
pmfn = _mfn(sp->up >> PAGE_SHIFT);
ASSERT(mfn_valid(pmfn));
@@ -2483,7 +2483,7 @@ static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
ASSERT(vaddr);
vaddr += sp->up & (PAGE_SIZE-1);
ASSERT(l1e_get_pfn(*(l1_pgentry_t *)vaddr) == mfn_x(smfn));
-
+
/* Is this the only reference to this shadow? */
rc = (sp->u.sh.count == 1) ? 1 : 0;
@@ -2508,7 +2508,7 @@ static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
break;
default: BUG(); /* Some wierd unknown shadow type */
}
-
+
sh_unmap_domain_page(vaddr);
if ( rc )
perfc_incr(shadow_up_pointer);
@@ -2519,8 +2519,8 @@ static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
}
void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
-/* Remove the shadows of this guest page.
- * If fast != 0, just try the quick heuristic, which will remove
+/* Remove the shadows of this guest page.
+ * If fast != 0, just try the quick heuristic, which will remove
* at most one reference to each shadow of the page. Otherwise, walk
* all the shadow tables looking for refs to shadows of this gmfn.
* If all != 0, kill the domain if we can't find all the shadows.
@@ -2530,7 +2530,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
struct page_info *pg = mfn_to_page(gmfn);
mfn_t smfn;
unsigned char t;
-
+
/* Dispatch table for getting per-type functions: each level must
* be called with the function to remove a lower-level shadow. */
static const hash_callback_t callbacks[SH_type_unused] = {
@@ -2642,7 +2642,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
domain_crash(v->domain);
}
- /* Need to flush TLBs now, so that linear maps are safe next time we
+ /* Need to flush TLBs now, so that linear maps are safe next time we
* take a fault. */
flush_tlb_mask(v->domain->domain_dirty_cpumask);
@@ -2656,18 +2656,18 @@ sh_remove_all_shadows_and_parents(struct vcpu *v, mfn_t gmfn)
{
sh_remove_shadows(v, gmfn, 0, 1);
/* XXX TODO:
- * Rework this hashtable walker to return a linked-list of all
- * the shadows it modified, then do breadth-first recursion
- * to find the way up to higher-level tables and unshadow them too.
+ * Rework this hashtable walker to return a linked-list of all
+ * the shadows it modified, then do breadth-first recursion
+ * to find the way up to higher-level tables and unshadow them too.
*
* The current code (just tearing down each page's shadows as we
- * detect that it is not a pagetable) is correct, but very slow.
+ * detect that it is not a pagetable) is correct, but very slow.
* It means extra emulated writes and slows down removal of mappings. */
}
/**************************************************************************/
-/* Reset the up-pointers of every L3 shadow to 0.
+/* Reset the up-pointers of every L3 shadow to 0.
* This is called when l3 shadows stop being pinnable, to clear out all
* the list-head bits so the up-pointer field is properly inititalised. */
static int sh_clear_up_pointer(struct vcpu *v, mfn_t smfn, mfn_t unused)
@@ -2711,7 +2711,7 @@ static void sh_update_paging_modes(struct vcpu *v)
ASSERT(paging_locked_by_me(d));
-#if (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB)
/* Make sure this vcpu has a virtual TLB array allocated */
if ( unlikely(!v->arch.paging.vtlb) )
{
@@ -2727,7 +2727,7 @@ static void sh_update_paging_modes(struct vcpu *v)
}
#endif /* (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB) */
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
if ( mfn_x(v->arch.paging.shadow.oos_snapshot[0]) == INVALID_MFN )
{
int i;
@@ -2768,7 +2768,7 @@ static void sh_update_paging_modes(struct vcpu *v)
ASSERT(shadow_mode_translate(d));
ASSERT(shadow_mode_external(d));
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Need to resync all our pages now, because if a page goes out
* of sync with paging enabled and is resynced with paging
* disabled, the resync will go wrong. */
@@ -2827,7 +2827,7 @@ static void sh_update_paging_modes(struct vcpu *v)
/* Need to make a new monitor table for the new mode */
mfn_t new_mfn, old_mfn;
- if ( v != current && vcpu_runnable(v) )
+ if ( v != current && vcpu_runnable(v) )
{
SHADOW_ERROR("Some third party (d=%u v=%u) is changing "
"this HVM vcpu's (d=%u v=%u) paging mode "
@@ -2847,7 +2847,7 @@ static void sh_update_paging_modes(struct vcpu *v)
SHADOW_PRINTK("new monitor table %"PRI_mfn "\n",
mfn_x(new_mfn));
- /* Don't be running on the old monitor table when we
+ /* Don't be running on the old monitor table when we
* pull it down! Switch CR3, and warn the HVM code that
* its host cr3 has changed. */
make_cr3(v, mfn_x(new_mfn));
@@ -2914,9 +2914,9 @@ static void sh_new_mode(struct domain *d, u32 new_mode)
int shadow_enable(struct domain *d, u32 mode)
/* Turn on "permanent" shadow features: external, translate, refcount.
* Can only be called once on a domain, and these features cannot be
- * disabled.
+ * disabled.
* Returns 0 for success, -errno for failure. */
-{
+{
unsigned int old_pages;
struct page_info *pg = NULL;
uint32_t *e;
@@ -2942,14 +2942,14 @@ int shadow_enable(struct domain *d, u32 mode)
if ( old_pages == 0 )
{
unsigned int r;
- paging_lock(d);
+ paging_lock(d);
r = sh_set_allocation(d, 1024, NULL); /* Use at least 4MB */
if ( r != 0 )
{
sh_set_allocation(d, 0, NULL);
rv = -ENOMEM;
goto out_locked;
- }
+ }
paging_unlock(d);
}
@@ -2957,7 +2957,7 @@ int shadow_enable(struct domain *d, u32 mode)
d->arch.paging.alloc_page = shadow_alloc_p2m_page;
d->arch.paging.free_page = shadow_free_p2m_page;
- /* Init the P2M table. Must be done before we take the paging lock
+ /* Init the P2M table. Must be done before we take the paging lock
* to avoid possible deadlock. */
if ( mode & PG_translate )
{
@@ -2970,7 +2970,7 @@ int shadow_enable(struct domain *d, u32 mode)
* have paging disabled */
if ( is_hvm_domain(d) )
{
- /* Get a single page from the shadow pool. Take it via the
+ /* Get a single page from the shadow pool. Take it via the
* P2M interface to make freeing it simpler afterwards. */
pg = shadow_alloc_p2m_page(d);
if ( pg == NULL )
@@ -2979,11 +2979,11 @@ int shadow_enable(struct domain *d, u32 mode)
goto out_unlocked;
}
/* Fill it with 32-bit, non-PAE superpage entries, each mapping 4MB
- * of virtual address space onto the same physical address range */
+ * of virtual address space onto the same physical address range */
e = __map_domain_page(pg);
for ( i = 0; i < PAGE_SIZE / sizeof(*e); i++ )
e[i] = ((0x400000U * i)
- | _PAGE_PRESENT | _PAGE_RW | _PAGE_USER
+ | _PAGE_PRESENT | _PAGE_RW | _PAGE_USER
| _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE);
sh_unmap_domain_page(e);
pg->u.inuse.type_info = PGT_l2_page_table | 1 | PGT_validated;
@@ -3005,8 +3005,8 @@ int shadow_enable(struct domain *d, u32 mode)
goto out_locked;
}
-#if (SHADOW_OPTIMIZATIONS & SHOPT_LINUX_L3_TOPLEVEL)
- /* We assume we're dealing with an older 64bit linux guest until we
+#if (SHADOW_OPTIMIZATIONS & SHOPT_LINUX_L3_TOPLEVEL)
+ /* We assume we're dealing with an older 64bit linux guest until we
* see the guest use more than one l4 per vcpu. */
d->arch.paging.shadow.opt_flags = SHOPT_LINUX_L3_TOPLEVEL;
#endif
@@ -3073,7 +3073,7 @@ void shadow_teardown(struct domain *d)
}
#endif /* (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB) */
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
{
int i;
mfn_t *oos_snapshot = v->arch.paging.shadow.oos_snapshot;
@@ -3093,24 +3093,24 @@ void shadow_teardown(struct domain *d)
SHADOW_PRINTK("teardown of domain %u starts."
" Shadow pages total = %u, free = %u, p2m=%u\n",
d->domain_id,
- d->arch.paging.shadow.total_pages,
- d->arch.paging.shadow.free_pages,
+ d->arch.paging.shadow.total_pages,
+ d->arch.paging.shadow.free_pages,
d->arch.paging.shadow.p2m_pages);
/* Destroy all the shadows and release memory to domheap */
sh_set_allocation(d, 0, NULL);
/* Release the hash table back to xenheap */
- if (d->arch.paging.shadow.hash_table)
+ if (d->arch.paging.shadow.hash_table)
shadow_hash_teardown(d);
/* Should not have any more memory held */
SHADOW_PRINTK("teardown done."
" Shadow pages total = %u, free = %u, p2m=%u\n",
- d->arch.paging.shadow.total_pages,
- d->arch.paging.shadow.free_pages,
+ d->arch.paging.shadow.total_pages,
+ d->arch.paging.shadow.free_pages,
d->arch.paging.shadow.p2m_pages);
ASSERT(d->arch.paging.shadow.total_pages == 0);
}
- /* Free the non-paged-vcpus pagetable; must happen after we've
+ /* Free the non-paged-vcpus pagetable; must happen after we've
* destroyed any shadows of it or sh_destroy_shadow will get confused. */
if ( !pagetable_is_null(d->arch.paging.shadow.unpaged_pagetable) )
{
@@ -3120,7 +3120,7 @@ void shadow_teardown(struct domain *d)
if ( !hvm_paging_enabled(v) )
v->arch.guest_table = pagetable_null();
}
- unpaged_pagetable =
+ unpaged_pagetable =
pagetable_get_page(d->arch.paging.shadow.unpaged_pagetable);
d->arch.paging.shadow.unpaged_pagetable = pagetable_null();
}
@@ -3140,7 +3140,7 @@ void shadow_teardown(struct domain *d)
paging_unlock(d);
/* Must be called outside the lock */
- if ( unpaged_pagetable )
+ if ( unpaged_pagetable )
shadow_free_p2m_page(d, unpaged_pagetable);
}
@@ -3150,11 +3150,11 @@ void shadow_final_teardown(struct domain *d)
SHADOW_PRINTK("dom %u final teardown starts."
" Shadow pages total = %u, free = %u, p2m=%u\n",
d->domain_id,
- d->arch.paging.shadow.total_pages,
- d->arch.paging.shadow.free_pages,
+ d->arch.paging.shadow.total_pages,
+ d->arch.paging.shadow.free_pages,
d->arch.paging.shadow.p2m_pages);
- /* Double-check that the domain didn't have any shadow memory.
+ /* Double-check that the domain didn't have any shadow memory.
* It is possible for a domain that never got domain_kill()ed
* to get here with its shadow allocation intact. */
if ( d->arch.paging.shadow.total_pages != 0 )
@@ -3168,8 +3168,8 @@ void shadow_final_teardown(struct domain *d)
SHADOW_PRINTK("dom %u final teardown done."
" Shadow pages total = %u, free = %u, p2m=%u\n",
d->domain_id,
- d->arch.paging.shadow.total_pages,
- d->arch.paging.shadow.free_pages,
+ d->arch.paging.shadow.total_pages,
+ d->arch.paging.shadow.free_pages,
d->arch.paging.shadow.p2m_pages);
paging_unlock(d);
}
@@ -3214,7 +3214,7 @@ static int shadow_one_bit_enable(struct domain *d, u32 mode)
return 0;
}
-static int shadow_one_bit_disable(struct domain *d, u32 mode)
+static int shadow_one_bit_disable(struct domain *d, u32 mode)
/* Turn off a single shadow mode feature */
{
struct vcpu *v;
@@ -3234,8 +3234,8 @@ static int shadow_one_bit_disable(struct domain *d, u32 mode)
SHADOW_PRINTK("un-shadowing of domain %u starts."
" Shadow pages total = %u, free = %u, p2m=%u\n",
d->domain_id,
- d->arch.paging.shadow.total_pages,
- d->arch.paging.shadow.free_pages,
+ d->arch.paging.shadow.total_pages,
+ d->arch.paging.shadow.free_pages,
d->arch.paging.shadow.p2m_pages);
for_each_vcpu(d, v)
{
@@ -3246,7 +3246,7 @@ static int shadow_one_bit_disable(struct domain *d, u32 mode)
else
make_cr3(v, pagetable_get_pfn(v->arch.guest_table));
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
{
int i;
mfn_t *oos_snapshot = v->arch.paging.shadow.oos_snapshot;
@@ -3267,8 +3267,8 @@ static int shadow_one_bit_disable(struct domain *d, u32 mode)
SHADOW_PRINTK("un-shadowing of domain %u done."
" Shadow pages total = %u, free = %u, p2m=%u\n",
d->domain_id,
- d->arch.paging.shadow.total_pages,
- d->arch.paging.shadow.free_pages,
+ d->arch.paging.shadow.total_pages,
+ d->arch.paging.shadow.free_pages,
d->arch.paging.shadow.p2m_pages);
}
@@ -3306,7 +3306,7 @@ static int shadow_test_disable(struct domain *d)
/* P2M map manipulations */
/* shadow specific code which should be called when P2M table entry is updated
- * with new content. It is responsible for update the entry, as well as other
+ * with new content. It is responsible for update the entry, as well as other
* shadow processing jobs.
*/
@@ -3329,7 +3329,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
{
mfn_t mfn = _mfn(l1e_get_pfn(*p));
p2m_type_t p2mt = p2m_flags_to_type(l1e_get_flags(*p));
- if ( (p2m_is_valid(p2mt) || p2m_is_grant(p2mt)) && mfn_valid(mfn) )
+ if ( (p2m_is_valid(p2mt) || p2m_is_grant(p2mt)) && mfn_valid(mfn) )
{
sh_remove_all_shadows_and_parents(v, mfn);
if ( sh_remove_all_mappings(v, mfn) )
@@ -3337,8 +3337,8 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
}
}
- /* If we're removing a superpage mapping from the p2m, we need to check
- * all the pages covered by it. If they're still there in the new
+ /* If we're removing a superpage mapping from the p2m, we need to check
+ * all the pages covered by it. If they're still there in the new
* scheme, that's OK, but otherwise they must be unshadowed. */
if ( level == 2 && (l1e_get_flags(*p) & _PAGE_PRESENT) &&
(l1e_get_flags(*p) & _PAGE_PSE) )
@@ -3355,13 +3355,13 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
/* If we're replacing a superpage with a normal L1 page, map it */
if ( (l1e_get_flags(new) & _PAGE_PRESENT)
- && !(l1e_get_flags(new) & _PAGE_PSE)
+ && !(l1e_get_flags(new) & _PAGE_PSE)
&& mfn_valid(nmfn) )
npte = map_domain_page(mfn_x(nmfn));
-
+
for ( i = 0; i < L1_PAGETABLE_ENTRIES; i++ )
{
- if ( !npte
+ if ( !npte
|| !p2m_is_ram(p2m_flags_to_type(l1e_get_flags(npte[i])))
|| l1e_get_pfn(npte[i]) != mfn_x(omfn) )
{
@@ -3374,7 +3374,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
omfn = _mfn(mfn_x(omfn) + 1);
}
flush_tlb_mask(&flushmask);
-
+
if ( npte )
unmap_domain_page(npte);
}
@@ -3389,7 +3389,7 @@ shadow_write_p2m_entry(struct domain *d, unsigned long gfn,
paging_lock(d);
/* If there are any shadows, update them. But if shadow_teardown()
- * has already been called then it's not safe to try. */
+ * has already been called then it's not safe to try. */
if ( likely(d->arch.paging.shadow.total_pages != 0) )
sh_unshadow_for_p2m_change(d, gfn, p, new, level);
@@ -3426,8 +3426,8 @@ static int sh_enable_log_dirty(struct domain *d, bool_t log_global)
paging_lock(d);
if ( shadow_mode_enabled(d) )
{
- /* This domain already has some shadows: need to clear them out
- * of the way to make sure that all references to guest memory are
+ /* This domain already has some shadows: need to clear them out
+ * of the way to make sure that all references to guest memory are
* properly write-protected */
shadow_blow_tables(d);
}
@@ -3439,7 +3439,7 @@ static int sh_enable_log_dirty(struct domain *d, bool_t log_global)
if ( is_pv_32on64_domain(d) )
d->arch.paging.shadow.opt_flags = SHOPT_LINUX_L3_TOPLEVEL;
#endif
-
+
ret = shadow_one_bit_enable(d, PG_log_dirty);
paging_unlock(d);
@@ -3454,12 +3454,12 @@ static int sh_disable_log_dirty(struct domain *d)
paging_lock(d);
ret = shadow_one_bit_disable(d, PG_log_dirty);
paging_unlock(d);
-
+
return ret;
}
-/* This function is called when we CLEAN log dirty bitmap. See
- * paging_log_dirty_op() for details.
+/* This function is called when we CLEAN log dirty bitmap. See
+ * paging_log_dirty_op() for details.
*/
static void sh_clean_dirty_bitmap(struct domain *d)
{
@@ -3519,7 +3519,7 @@ int shadow_track_dirty_vram(struct domain *d,
* no need to be careful. */
if ( !dirty_vram )
{
- /* Throw away all the shadows rather than walking through them
+ /* Throw away all the shadows rather than walking through them
* up to nr times getting rid of mappings of each pfn */
shadow_blow_tables(d);
@@ -3665,7 +3665,7 @@ out:
/**************************************************************************/
/* Shadow-control XEN_DOMCTL dispatcher */
-int shadow_domctl(struct domain *d,
+int shadow_domctl(struct domain *d,
xen_domctl_shadow_op_t *sc,
XEN_GUEST_HANDLE_PARAM(void) u_domctl)
{
@@ -3675,7 +3675,7 @@ int shadow_domctl(struct domain *d,
{
case XEN_DOMCTL_SHADOW_OP_OFF:
if ( d->arch.paging.mode == PG_SH_enable )
- if ( (rc = shadow_test_disable(d)) != 0 )
+ if ( (rc = shadow_test_disable(d)) != 0 )
return rc;
return 0;
@@ -3695,7 +3695,7 @@ int shadow_domctl(struct domain *d,
case XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION:
paging_lock(d);
if ( sc->mb == 0 && shadow_mode_enabled(d) )
- {
+ {
/* Can't set the allocation to zero unless the domain stops using
* shadow pagetables first */
SHADOW_ERROR("Can't set shadow allocation to zero, domain %u"
@@ -3709,7 +3709,7 @@ int shadow_domctl(struct domain *d,
/* Not finished. Set up to re-run the call. */
rc = hypercall_create_continuation(
__HYPERVISOR_domctl, "h", u_domctl);
- else
+ else
/* Finished. Return the new allocation */
sc->mb = shadow_get_allocation(d);
return rc;
@@ -3726,7 +3726,7 @@ int shadow_domctl(struct domain *d,
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES_FULL
-void shadow_audit_tables(struct vcpu *v)
+void shadow_audit_tables(struct vcpu *v)
{
/* Dispatch table for getting per-type functions */
static const hash_callback_t callbacks[SH_type_unused] = {
@@ -3746,7 +3746,7 @@ void shadow_audit_tables(struct vcpu *v)
SHADOW_INTERNAL_NAME(sh_audit_l4_table, 4), /* l4_64 */
NULL /* All the rest */
};
- unsigned int mask;
+ unsigned int mask;
if ( !(SHADOW_AUDIT_ENABLE) )
return;
@@ -3765,7 +3765,7 @@ void shadow_audit_tables(struct vcpu *v)
case 2: mask = (SHF_L1_32|SHF_FL1_32|SHF_L2_32); break;
case 3: mask = (SHF_L1_PAE|SHF_FL1_PAE|SHF_L2_PAE
|SHF_L2H_PAE); break;
- case 4: mask = (SHF_L1_64|SHF_FL1_64|SHF_L2_64
+ case 4: mask = (SHF_L1_64|SHF_FL1_64|SHF_L2_64
|SHF_L3_64|SHF_L4_64); break;
default: BUG();
}
@@ -3782,5 +3782,5 @@ void shadow_audit_tables(struct vcpu *v)
* c-file-style: "BSD"
* c-basic-offset: 4
* indent-tabs-mode: nil
- * End:
+ * End:
*/
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 5fc10c9..434df61 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -1,7 +1,7 @@
/******************************************************************************
* arch/x86/mm/shadow/multi.c
*
- * Simple, mostly-synchronous shadow page tables.
+ * Simple, mostly-synchronous shadow page tables.
* Parts of this code are Copyright (c) 2006 by XenSource Inc.
* Parts of this code are Copyright (c) 2006 by Michael A Fetterman
* Parts based on earlier work by Michael A Fetterman, Ian Pratt et al.
@@ -43,25 +43,25 @@
#include "types.h"
/* THINGS TO DO LATER:
- *
+ *
* TEARDOWN HEURISTICS
- * Also: have a heuristic for when to destroy a previous paging-mode's
+ * Also: have a heuristic for when to destroy a previous paging-mode's
* shadows. When a guest is done with its start-of-day 32-bit tables
- * and reuses the memory we want to drop those shadows. Start with
- * shadows in a page in two modes as a hint, but beware of clever tricks
+ * and reuses the memory we want to drop those shadows. Start with
+ * shadows in a page in two modes as a hint, but beware of clever tricks
* like reusing a pagetable for both PAE and 64-bit during boot...
*
* PAE LINEAR MAPS
* Rework shadow_get_l*e() to have the option of using map_domain_page()
- * instead of linear maps. Add appropriate unmap_l*e calls in the users.
- * Then we can test the speed difference made by linear maps. If the
- * map_domain_page() version is OK on PAE, we could maybe allow a lightweight
- * l3-and-l2h-only shadow mode for PAE PV guests that would allow them
- * to share l2h pages again.
+ * instead of linear maps. Add appropriate unmap_l*e calls in the users.
+ * Then we can test the speed difference made by linear maps. If the
+ * map_domain_page() version is OK on PAE, we could maybe allow a lightweight
+ * l3-and-l2h-only shadow mode for PAE PV guests that would allow them
+ * to share l2h pages again.
*
* PSE disabled / PSE36
* We don't support any modes other than PSE enabled, PSE36 disabled.
- * Neither of those would be hard to change, but we'd need to be able to
+ * Neither of those would be hard to change, but we'd need to be able to
* deal with shadows made in one mode and used in another.
*/
@@ -90,7 +90,7 @@ static char *fetch_type_names[] = {
* shadow L1 which maps its "splinters".
*/
-static inline mfn_t
+static inline mfn_t
get_fl1_shadow_status(struct vcpu *v, gfn_t gfn)
/* Look for FL1 shadows in the hash table */
{
@@ -99,7 +99,7 @@ get_fl1_shadow_status(struct vcpu *v, gfn_t gfn)
return smfn;
}
-static inline mfn_t
+static inline mfn_t
get_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
/* Look for shadows in the hash table */
{
@@ -109,7 +109,7 @@ get_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
return smfn;
}
-static inline void
+static inline void
set_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
/* Put an FL1 shadow into the hash table */
{
@@ -120,7 +120,7 @@ set_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
shadow_hash_insert(v, gfn_x(gfn), SH_type_fl1_shadow, smfn);
}
-static inline void
+static inline void
set_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
/* Put a shadow into the hash table */
{
@@ -143,7 +143,7 @@ set_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
shadow_hash_insert(v, mfn_x(gmfn), shadow_type, smfn);
}
-static inline void
+static inline void
delete_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
/* Remove a shadow from the hash table */
{
@@ -153,7 +153,7 @@ delete_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
shadow_hash_delete(v, gfn_x(gfn), SH_type_fl1_shadow, smfn);
}
-static inline void
+static inline void
delete_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
/* Remove a shadow from the hash table */
{
@@ -172,10 +172,10 @@ delete_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
/* Functions for walking the guest page tables */
static inline uint32_t
-sh_walk_guest_tables(struct vcpu *v, unsigned long va, walk_t *gw,
+sh_walk_guest_tables(struct vcpu *v, unsigned long va, walk_t *gw,
uint32_t pfec)
{
- return guest_walk_tables(v, p2m_get_hostp2m(v->domain), va, gw, pfec,
+ return guest_walk_tables(v, p2m_get_hostp2m(v->domain), va, gw, pfec,
#if GUEST_PAGING_LEVELS == 3 /* PAE */
_mfn(INVALID_MFN),
v->arch.paging.shadow.gl3e
@@ -323,7 +323,7 @@ gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES
/* Lightweight audit: pass all the shadows associated with this guest walk
* through the audit mechanisms */
-static void sh_audit_gw(struct vcpu *v, walk_t *gw)
+static void sh_audit_gw(struct vcpu *v, walk_t *gw)
{
mfn_t smfn;
@@ -332,32 +332,32 @@ static void sh_audit_gw(struct vcpu *v, walk_t *gw)
#if GUEST_PAGING_LEVELS >= 4 /* 64-bit only... */
if ( mfn_valid(gw->l4mfn)
- && mfn_valid((smfn = get_shadow_status(v, gw->l4mfn,
+ && mfn_valid((smfn = get_shadow_status(v, gw->l4mfn,
SH_type_l4_shadow))) )
(void) sh_audit_l4_table(v, smfn, _mfn(INVALID_MFN));
if ( mfn_valid(gw->l3mfn)
- && mfn_valid((smfn = get_shadow_status(v, gw->l3mfn,
+ && mfn_valid((smfn = get_shadow_status(v, gw->l3mfn,
SH_type_l3_shadow))) )
(void) sh_audit_l3_table(v, smfn, _mfn(INVALID_MFN));
#endif /* PAE or 64... */
if ( mfn_valid(gw->l2mfn) )
{
- if ( mfn_valid((smfn = get_shadow_status(v, gw->l2mfn,
+ if ( mfn_valid((smfn = get_shadow_status(v, gw->l2mfn,
SH_type_l2_shadow))) )
(void) sh_audit_l2_table(v, smfn, _mfn(INVALID_MFN));
#if GUEST_PAGING_LEVELS == 3
- if ( mfn_valid((smfn = get_shadow_status(v, gw->l2mfn,
+ if ( mfn_valid((smfn = get_shadow_status(v, gw->l2mfn,
SH_type_l2h_shadow))) )
(void) sh_audit_l2_table(v, smfn, _mfn(INVALID_MFN));
#endif
}
if ( mfn_valid(gw->l1mfn)
- && mfn_valid((smfn = get_shadow_status(v, gw->l1mfn,
+ && mfn_valid((smfn = get_shadow_status(v, gw->l1mfn,
SH_type_l1_shadow))) )
(void) sh_audit_l1_table(v, smfn, _mfn(INVALID_MFN));
else if ( (guest_l2e_get_flags(gw->l2e) & _PAGE_PRESENT)
&& (guest_l2e_get_flags(gw->l2e) & _PAGE_PSE)
- && mfn_valid(
+ && mfn_valid(
(smfn = get_fl1_shadow_status(v, guest_l2e_get_gfn(gw->l2e)))) )
(void) sh_audit_fl1_table(v, smfn, _mfn(INVALID_MFN));
}
@@ -376,11 +376,11 @@ sh_guest_map_l1e(struct vcpu *v, unsigned long addr,
walk_t gw;
ASSERT(shadow_mode_translate(v->domain));
-
+
// XXX -- this is expensive, but it's easy to cobble together...
// FIXME!
- if ( sh_walk_guest_tables(v, addr, &gw, PFEC_page_present) == 0
+ if ( sh_walk_guest_tables(v, addr, &gw, PFEC_page_present) == 0
&& mfn_valid(gw.l1mfn) )
{
if ( gl1mfn )
@@ -398,7 +398,7 @@ sh_guest_get_eff_l1e(struct vcpu *v, unsigned long addr, void *eff_l1e)
walk_t gw;
ASSERT(shadow_mode_translate(v->domain));
-
+
// XXX -- this is expensive, but it's easy to cobble together...
// FIXME!
@@ -506,12 +506,12 @@ shadow_l4_index(mfn_t *smfn, u32 guest_index)
*/
static always_inline void
-_sh_propagate(struct vcpu *v,
+_sh_propagate(struct vcpu *v,
guest_intpte_t guest_intpte,
- mfn_t target_mfn,
+ mfn_t target_mfn,
void *shadow_entry_ptr,
int level,
- fetch_type_t ft,
+ fetch_type_t ft,
p2m_type_t p2mt)
{
guest_l1e_t guest_entry = { guest_intpte };
@@ -537,11 +537,11 @@ _sh_propagate(struct vcpu *v,
if ( unlikely(!(gflags & _PAGE_PRESENT)) )
{
#if !(SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
- /* If a guest l1 entry is not present, shadow with the magic
+ /* If a guest l1 entry is not present, shadow with the magic
* guest-not-present entry. */
if ( level == 1 )
*sp = sh_l1e_gnp();
- else
+ else
#endif /* !OOS */
*sp = shadow_l1e_empty();
goto done;
@@ -562,7 +562,7 @@ _sh_propagate(struct vcpu *v,
// return early.
//
if ( !mfn_valid(target_mfn)
- && !(level == 1 && (!shadow_mode_refcounts(d)
+ && !(level == 1 && (!shadow_mode_refcounts(d)
|| p2mt == p2m_mmio_direct)) )
{
ASSERT((ft == ft_prefetch));
@@ -595,7 +595,7 @@ _sh_propagate(struct vcpu *v,
ASSERT(!(sflags & (_PAGE_PAT | _PAGE_PCD | _PAGE_PWT)));
/* compute the PAT index for shadow page entry when VT-d is enabled
- * and device assigned.
+ * and device assigned.
* 1) direct MMIO: compute the PAT index with gMTRR=UC and gPAT.
* 2) if enables snoop control, compute the PAT index as WB.
* 3) if disables snoop control, compute the PAT index with
@@ -613,7 +613,7 @@ _sh_propagate(struct vcpu *v,
gflags,
gfn_to_paddr(target_gfn),
pfn_to_paddr(mfn_x(target_mfn)),
- MTRR_TYPE_UNCACHABLE);
+ MTRR_TYPE_UNCACHABLE);
else if ( iommu_snoop )
sflags |= pat_type_2_pte_flags(PAT_TYPE_WRBACK);
else
@@ -654,12 +654,12 @@ _sh_propagate(struct vcpu *v,
// Only allow the guest write access to a page a) on a demand fault,
// or b) if the page is already marked as dirty.
//
- // (We handle log-dirty entirely inside the shadow code, without using the
+ // (We handle log-dirty entirely inside the shadow code, without using the
// p2m_ram_logdirty p2m type: only HAP uses that.)
if ( unlikely((level == 1) && shadow_mode_log_dirty(d)) )
{
if ( mfn_valid(target_mfn) ) {
- if ( ft & FETCH_TYPE_WRITE )
+ if ( ft & FETCH_TYPE_WRITE )
paging_mark_dirty(d, mfn_x(target_mfn));
else if ( !paging_mfn_is_dirty(d, target_mfn) )
sflags &= ~_PAGE_RW;
@@ -682,10 +682,10 @@ _sh_propagate(struct vcpu *v,
(p2mt == p2m_mmio_direct &&
rangeset_contains_singleton(mmio_ro_ranges, mfn_x(target_mfn))) )
sflags &= ~_PAGE_RW;
-
+
// protect guest page tables
//
- if ( unlikely((level == 1)
+ if ( unlikely((level == 1)
&& sh_mfn_is_a_page_table(target_mfn)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC )
/* Unless the page is out of sync and the guest is
@@ -699,7 +699,7 @@ _sh_propagate(struct vcpu *v,
// PV guests in 64-bit mode use two different page tables for user vs
// supervisor permissions, making the guest's _PAGE_USER bit irrelevant.
// It is always shadowed as present...
- if ( (GUEST_PAGING_LEVELS == 4) && !is_pv_32on64_domain(d)
+ if ( (GUEST_PAGING_LEVELS == 4) && !is_pv_32on64_domain(d)
&& is_pv_domain(d) )
{
sflags |= _PAGE_USER;
@@ -720,7 +720,7 @@ _sh_propagate(struct vcpu *v,
#if GUEST_PAGING_LEVELS >= 4
static void
-l4e_propagate_from_guest(struct vcpu *v,
+l4e_propagate_from_guest(struct vcpu *v,
guest_l4e_t gl4e,
mfn_t sl3mfn,
shadow_l4e_t *sl4e,
@@ -732,7 +732,7 @@ l4e_propagate_from_guest(struct vcpu *v,
static void
l3e_propagate_from_guest(struct vcpu *v,
guest_l3e_t gl3e,
- mfn_t sl2mfn,
+ mfn_t sl2mfn,
shadow_l3e_t *sl3e,
fetch_type_t ft)
{
@@ -741,7 +741,7 @@ l3e_propagate_from_guest(struct vcpu *v,
#endif // GUEST_PAGING_LEVELS >= 4
static void
-l2e_propagate_from_guest(struct vcpu *v,
+l2e_propagate_from_guest(struct vcpu *v,
guest_l2e_t gl2e,
mfn_t sl1mfn,
shadow_l2e_t *sl2e,
@@ -751,11 +751,11 @@ l2e_propagate_from_guest(struct vcpu *v,
}
static void
-l1e_propagate_from_guest(struct vcpu *v,
+l1e_propagate_from_guest(struct vcpu *v,
guest_l1e_t gl1e,
- mfn_t gmfn,
+ mfn_t gmfn,
shadow_l1e_t *sl1e,
- fetch_type_t ft,
+ fetch_type_t ft,
p2m_type_t p2mt)
{
_sh_propagate(v, gl1e.l1, gmfn, sl1e, 1, ft, p2mt);
@@ -768,10 +768,10 @@ l1e_propagate_from_guest(struct vcpu *v,
* functions which ever write (non-zero) data onto a shadow page.
*/
-static inline void safe_write_entry(void *dst, void *src)
+static inline void safe_write_entry(void *dst, void *src)
/* Copy one PTE safely when processors might be running on the
* destination pagetable. This does *not* give safety against
- * concurrent writes (that's what the paging lock is for), just
+ * concurrent writes (that's what the paging lock is for), just
* stops the hardware picking up partially written entries. */
{
volatile unsigned long *d = dst;
@@ -784,7 +784,7 @@ static inline void safe_write_entry(void *dst, void *src)
}
-static inline void
+static inline void
shadow_write_entries(void *d, void *s, int entries, mfn_t mfn)
/* This function does the actual writes to shadow pages.
* It must not be called directly, since it doesn't do the bookkeeping
@@ -797,10 +797,10 @@ shadow_write_entries(void *d, void *s, int entries, mfn_t mfn)
/* Because we mirror access rights at all levels in the shadow, an
* l2 (or higher) entry with the RW bit cleared will leave us with
- * no write access through the linear map.
- * We detect that by writing to the shadow with copy_to_user() and
+ * no write access through the linear map.
+ * We detect that by writing to the shadow with copy_to_user() and
* using map_domain_page() to get a writeable mapping if we need to. */
- if ( __copy_to_user(d, d, sizeof (unsigned long)) != 0 )
+ if ( __copy_to_user(d, d, sizeof (unsigned long)) != 0 )
{
perfc_incr(shadow_linear_map_failed);
map = sh_map_domain_page(mfn);
@@ -874,7 +874,7 @@ shadow_get_page_from_l1e(shadow_l1e_t sl1e, struct domain *d, p2m_type_t type)
static void inline
shadow_put_page_from_l1e(shadow_l1e_t sl1e, struct domain *d)
-{
+{
if ( !shadow_mode_refcounts(d) )
return;
@@ -882,9 +882,9 @@ shadow_put_page_from_l1e(shadow_l1e_t sl1e, struct domain *d)
}
#if GUEST_PAGING_LEVELS >= 4
-static int shadow_set_l4e(struct vcpu *v,
- shadow_l4e_t *sl4e,
- shadow_l4e_t new_sl4e,
+static int shadow_set_l4e(struct vcpu *v,
+ shadow_l4e_t *sl4e,
+ shadow_l4e_t new_sl4e,
mfn_t sl4mfn)
{
int flags = 0, ok;
@@ -894,13 +894,13 @@ static int shadow_set_l4e(struct vcpu *v,
old_sl4e = *sl4e;
if ( old_sl4e.l4 == new_sl4e.l4 ) return 0; /* Nothing to do */
-
- paddr = ((((paddr_t)mfn_x(sl4mfn)) << PAGE_SHIFT)
+
+ paddr = ((((paddr_t)mfn_x(sl4mfn)) << PAGE_SHIFT)
| (((unsigned long)sl4e) & ~PAGE_MASK));
- if ( shadow_l4e_get_flags(new_sl4e) & _PAGE_PRESENT )
+ if ( shadow_l4e_get_flags(new_sl4e) & _PAGE_PRESENT )
{
- /* About to install a new reference */
+ /* About to install a new reference */
mfn_t sl3mfn = shadow_l4e_get_mfn(new_sl4e);
ok = sh_get_ref(v, sl3mfn, paddr);
/* Are we pinning l3 shadows to handle wierd linux behaviour? */
@@ -917,12 +917,12 @@ static int shadow_set_l4e(struct vcpu *v,
shadow_write_entries(sl4e, &new_sl4e, 1, sl4mfn);
flags |= SHADOW_SET_CHANGED;
- if ( shadow_l4e_get_flags(old_sl4e) & _PAGE_PRESENT )
+ if ( shadow_l4e_get_flags(old_sl4e) & _PAGE_PRESENT )
{
/* We lost a reference to an old mfn. */
mfn_t osl3mfn = shadow_l4e_get_mfn(old_sl4e);
if ( (mfn_x(osl3mfn) != mfn_x(shadow_l4e_get_mfn(new_sl4e)))
- || !perms_strictly_increased(shadow_l4e_get_flags(old_sl4e),
+ || !perms_strictly_increased(shadow_l4e_get_flags(old_sl4e),
shadow_l4e_get_flags(new_sl4e)) )
{
flags |= SHADOW_SET_FLUSH;
@@ -932,9 +932,9 @@ static int shadow_set_l4e(struct vcpu *v,
return flags;
}
-static int shadow_set_l3e(struct vcpu *v,
- shadow_l3e_t *sl3e,
- shadow_l3e_t new_sl3e,
+static int shadow_set_l3e(struct vcpu *v,
+ shadow_l3e_t *sl3e,
+ shadow_l3e_t new_sl3e,
mfn_t sl3mfn)
{
int flags = 0;
@@ -945,12 +945,12 @@ static int shadow_set_l3e(struct vcpu *v,
if ( old_sl3e.l3 == new_sl3e.l3 ) return 0; /* Nothing to do */
- paddr = ((((paddr_t)mfn_x(sl3mfn)) << PAGE_SHIFT)
+ paddr = ((((paddr_t)mfn_x(sl3mfn)) << PAGE_SHIFT)
| (((unsigned long)sl3e) & ~PAGE_MASK));
-
+
if ( shadow_l3e_get_flags(new_sl3e) & _PAGE_PRESENT )
{
- /* About to install a new reference */
+ /* About to install a new reference */
if ( !sh_get_ref(v, shadow_l3e_get_mfn(new_sl3e), paddr) )
{
domain_crash(v->domain);
@@ -962,13 +962,13 @@ static int shadow_set_l3e(struct vcpu *v,
shadow_write_entries(sl3e, &new_sl3e, 1, sl3mfn);
flags |= SHADOW_SET_CHANGED;
- if ( shadow_l3e_get_flags(old_sl3e) & _PAGE_PRESENT )
+ if ( shadow_l3e_get_flags(old_sl3e) & _PAGE_PRESENT )
{
/* We lost a reference to an old mfn. */
mfn_t osl2mfn = shadow_l3e_get_mfn(old_sl3e);
if ( (mfn_x(osl2mfn) != mfn_x(shadow_l3e_get_mfn(new_sl3e))) ||
- !perms_strictly_increased(shadow_l3e_get_flags(old_sl3e),
- shadow_l3e_get_flags(new_sl3e)) )
+ !perms_strictly_increased(shadow_l3e_get_flags(old_sl3e),
+ shadow_l3e_get_flags(new_sl3e)) )
{
flags |= SHADOW_SET_FLUSH;
}
@@ -976,11 +976,11 @@ static int shadow_set_l3e(struct vcpu *v,
}
return flags;
}
-#endif /* GUEST_PAGING_LEVELS >= 4 */
+#endif /* GUEST_PAGING_LEVELS >= 4 */
-static int shadow_set_l2e(struct vcpu *v,
- shadow_l2e_t *sl2e,
- shadow_l2e_t new_sl2e,
+static int shadow_set_l2e(struct vcpu *v,
+ shadow_l2e_t *sl2e,
+ shadow_l2e_t new_sl2e,
mfn_t sl2mfn)
{
int flags = 0;
@@ -990,7 +990,7 @@ static int shadow_set_l2e(struct vcpu *v,
#if GUEST_PAGING_LEVELS == 2
/* In 2-on-3 we work with pairs of l2es pointing at two-page
* shadows. Reference counting and up-pointers track from the first
- * page of the shadow to the first l2e, so make sure that we're
+ * page of the shadow to the first l2e, so make sure that we're
* working with those:
* Start with a pair of identical entries */
shadow_l2e_t pair[2] = { new_sl2e, new_sl2e };
@@ -1000,13 +1000,13 @@ static int shadow_set_l2e(struct vcpu *v,
ASSERT(sl2e != NULL);
old_sl2e = *sl2e;
-
+
if ( old_sl2e.l2 == new_sl2e.l2 ) return 0; /* Nothing to do */
-
+
paddr = ((((paddr_t)mfn_x(sl2mfn)) << PAGE_SHIFT)
| (((unsigned long)sl2e) & ~PAGE_MASK));
- if ( shadow_l2e_get_flags(new_sl2e) & _PAGE_PRESENT )
+ if ( shadow_l2e_get_flags(new_sl2e) & _PAGE_PRESENT )
{
mfn_t sl1mfn = shadow_l2e_get_mfn(new_sl2e);
ASSERT(mfn_to_page(sl1mfn)->u.sh.head);
@@ -1028,7 +1028,7 @@ static int shadow_set_l2e(struct vcpu *v,
the GFN instead of the GMFN, and it's definitely not
OOS. */
if ( (sp->u.sh.type != SH_type_fl1_shadow) && mfn_valid(gl1mfn)
- && mfn_is_out_of_sync(gl1mfn) )
+ && mfn_is_out_of_sync(gl1mfn) )
sh_resync(v, gl1mfn);
}
#endif
@@ -1047,13 +1047,13 @@ static int shadow_set_l2e(struct vcpu *v,
#endif
flags |= SHADOW_SET_CHANGED;
- if ( shadow_l2e_get_flags(old_sl2e) & _PAGE_PRESENT )
+ if ( shadow_l2e_get_flags(old_sl2e) & _PAGE_PRESENT )
{
/* We lost a reference to an old mfn. */
mfn_t osl1mfn = shadow_l2e_get_mfn(old_sl2e);
if ( (mfn_x(osl1mfn) != mfn_x(shadow_l2e_get_mfn(new_sl2e))) ||
- !perms_strictly_increased(shadow_l2e_get_flags(old_sl2e),
- shadow_l2e_get_flags(new_sl2e)) )
+ !perms_strictly_increased(shadow_l2e_get_flags(old_sl2e),
+ shadow_l2e_get_flags(new_sl2e)) )
{
flags |= SHADOW_SET_FLUSH;
}
@@ -1066,7 +1066,7 @@ static inline void shadow_vram_get_l1e(shadow_l1e_t new_sl1e,
shadow_l1e_t *sl1e,
mfn_t sl1mfn,
struct domain *d)
-{
+{
mfn_t mfn = shadow_l1e_get_mfn(new_sl1e);
int flags = shadow_l1e_get_flags(new_sl1e);
unsigned long gfn;
@@ -1085,7 +1085,7 @@ static inline void shadow_vram_get_l1e(shadow_l1e_t new_sl1e,
{
unsigned long i = gfn - dirty_vram->begin_pfn;
struct page_info *page = mfn_to_page(mfn);
-
+
if ( (page->u.inuse.type_info & PGT_count_mask) == 1 )
/* Initial guest reference, record it */
dirty_vram->sl1ma[i] = pfn_to_paddr(mfn_x(sl1mfn))
@@ -1159,8 +1159,8 @@ static inline void shadow_vram_put_l1e(shadow_l1e_t old_sl1e,
}
}
-static int shadow_set_l1e(struct vcpu *v,
- shadow_l1e_t *sl1e,
+static int shadow_set_l1e(struct vcpu *v,
+ shadow_l1e_t *sl1e,
shadow_l1e_t new_sl1e,
p2m_type_t new_type,
mfn_t sl1mfn)
@@ -1179,15 +1179,15 @@ static int shadow_set_l1e(struct vcpu *v,
== (_PAGE_RW|_PAGE_PRESENT)) )
oos_fixup_add(v, new_gmfn, sl1mfn, pgentry_ptr_to_slot(sl1e));
#endif
-
+
old_sl1e = *sl1e;
if ( old_sl1e.l1 == new_sl1e.l1 ) return 0; /* Nothing to do */
-
+
if ( (shadow_l1e_get_flags(new_sl1e) & _PAGE_PRESENT)
- && !sh_l1e_is_magic(new_sl1e) )
+ && !sh_l1e_is_magic(new_sl1e) )
{
- /* About to install a new reference */
+ /* About to install a new reference */
if ( shadow_mode_refcounts(d) ) {
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_SHADOW_L1_GET_REF);
switch ( shadow_get_page_from_l1e(new_sl1e, d, new_type) )
@@ -1205,45 +1205,45 @@ static int shadow_set_l1e(struct vcpu *v,
break;
}
}
- }
+ }
/* Write the new entry */
shadow_write_entries(sl1e, &new_sl1e, 1, sl1mfn);
flags |= SHADOW_SET_CHANGED;
- if ( (shadow_l1e_get_flags(old_sl1e) & _PAGE_PRESENT)
+ if ( (shadow_l1e_get_flags(old_sl1e) & _PAGE_PRESENT)
&& !sh_l1e_is_magic(old_sl1e) )
{
/* We lost a reference to an old mfn. */
- /* N.B. Unlike higher-level sets, never need an extra flush
- * when writing an l1e. Because it points to the same guest frame
+ /* N.B. Unlike higher-level sets, never need an extra flush
+ * when writing an l1e. Because it points to the same guest frame
* as the guest l1e did, it's the guest's responsibility to
* trigger a flush later. */
- if ( shadow_mode_refcounts(d) )
+ if ( shadow_mode_refcounts(d) )
{
shadow_vram_put_l1e(old_sl1e, sl1e, sl1mfn, d);
shadow_put_page_from_l1e(old_sl1e, d);
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_SHADOW_L1_PUT_REF);
- }
+ }
}
return flags;
}
/**************************************************************************/
-/* Macros to walk pagetables. These take the shadow of a pagetable and
- * walk every "interesting" entry. That is, they don't touch Xen mappings,
- * and for 32-bit l2s shadowed onto PAE or 64-bit, they only touch every
+/* Macros to walk pagetables. These take the shadow of a pagetable and
+ * walk every "interesting" entry. That is, they don't touch Xen mappings,
+ * and for 32-bit l2s shadowed onto PAE or 64-bit, they only touch every
* second entry (since pairs of entries are managed together). For multi-page
* shadows they walk all pages.
- *
- * Arguments are an MFN, the variable to point to each entry, a variable
- * to indicate that we are done (we will shortcut to the end of the scan
+ *
+ * Arguments are an MFN, the variable to point to each entry, a variable
+ * to indicate that we are done (we will shortcut to the end of the scan
* when _done != 0), a variable to indicate that we should avoid Xen mappings,
- * and the code.
+ * and the code.
*
- * WARNING: These macros have side-effects. They change the values of both
- * the pointer and the MFN. */
+ * WARNING: These macros have side-effects. They change the values of both
+ * the pointer and the MFN. */
static inline void increment_ptr_to_guest_entry(void *ptr)
{
@@ -1288,7 +1288,7 @@ do { \
#define SHADOW_FOREACH_L1E(_sl1mfn, _sl1e, _gl1p, _done, _code) \
_SHADOW_FOREACH_L1E(_sl1mfn, _sl1e, _gl1p, _done, _code)
#endif
-
+
#if GUEST_PAGING_LEVELS == 2
@@ -1335,7 +1335,7 @@ do { \
sh_unmap_domain_page(_sp); \
} while (0)
-#else
+#else
/* 64-bit l2: touch all entries except for PAE compat guests. */
#define SHADOW_FOREACH_L2E(_sl2mfn, _sl2e, _gl2p, _done, _dom, _code) \
@@ -1424,7 +1424,7 @@ void sh_install_xen_entries_in_l4(struct vcpu *v, mfn_t gl4mfn, mfn_t sl4mfn)
sl4e = sh_map_domain_page(sl4mfn);
BUILD_BUG_ON(sizeof (l4_pgentry_t) != sizeof (shadow_l4e_t));
-
+
/* Copy the common Xen mappings from the idle domain */
slots = (shadow_mode_external(d)
? ROOT_PAGETABLE_XEN_SLOTS
@@ -1458,7 +1458,7 @@ void sh_install_xen_entries_in_l4(struct vcpu *v, mfn_t gl4mfn, mfn_t sl4mfn)
shadow_l4e_from_mfn(gl4mfn, __PAGE_HYPERVISOR);
}
- sh_unmap_domain_page(sl4e);
+ sh_unmap_domain_page(sl4e);
}
#endif
@@ -1504,12 +1504,12 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
mfn_to_page(smfn)->up = 0;
#if GUEST_PAGING_LEVELS == 4
-#if (SHADOW_OPTIMIZATIONS & SHOPT_LINUX_L3_TOPLEVEL)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_LINUX_L3_TOPLEVEL)
if ( shadow_type == SH_type_l4_64_shadow &&
unlikely(v->domain->arch.paging.shadow.opt_flags & SHOPT_LINUX_L3_TOPLEVEL) )
{
/* We're shadowing a new l4, but we've been assuming the guest uses
- * only one l4 per vcpu and context switches using an l4 entry.
+ * only one l4 per vcpu and context switches using an l4 entry.
* Count the number of active l4 shadows. If there are enough
* of them, decide that this isn't an old linux guest, and stop
* pinning l3es. This is not very quick but it doesn't happen
@@ -1522,9 +1522,9 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
if ( sp->u.sh.type == SH_type_l4_64_shadow )
l4count++;
}
- for_each_vcpu ( v->domain, v2 )
+ for_each_vcpu ( v->domain, v2 )
vcpus++;
- if ( l4count > 2 * vcpus )
+ if ( l4count > 2 * vcpus )
{
/* Unpin all the pinned l3 tables, and don't pin any more. */
page_list_for_each_safe(sp, t, &v->domain->arch.paging.shadow.pinned_shadows)
@@ -1542,7 +1542,7 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
// Create the Xen mappings...
if ( !shadow_mode_external(v->domain) )
{
- switch (shadow_type)
+ switch (shadow_type)
{
#if GUEST_PAGING_LEVELS == 4
case SH_type_l4_shadow:
@@ -1584,7 +1584,7 @@ sh_make_monitor_table(struct vcpu *v)
struct domain *d = v->domain;
ASSERT(pagetable_get_pfn(v->arch.monitor_table) == 0);
-
+
/* Guarantee we can get the memory we need */
shadow_prealloc(d, SH_type_monitor_table, CONFIG_PAGING_LEVELS);
@@ -1599,8 +1599,8 @@ sh_make_monitor_table(struct vcpu *v)
mfn_t m3mfn, m2mfn;
l4_pgentry_t *l4e;
l3_pgentry_t *l3e;
- /* Install an l3 table and an l2 table that will hold the shadow
- * linear map entries. This overrides the linear map entry that
+ /* Install an l3 table and an l2 table that will hold the shadow
+ * linear map entries. This overrides the linear map entry that
* was installed by sh_install_xen_entries_in_l4. */
l4e = sh_map_domain_page(m4mfn);
@@ -1622,7 +1622,7 @@ sh_make_monitor_table(struct vcpu *v)
m3mfn = shadow_alloc(d, SH_type_monitor_table, 0);
mfn_to_page(m3mfn)->shadow_flags = 3;
l4e[0] = l4e_from_pfn(mfn_x(m3mfn), __PAGE_HYPERVISOR);
-
+
m2mfn = shadow_alloc(d, SH_type_monitor_table, 0);
mfn_to_page(m2mfn)->shadow_flags = 2;
l3e = sh_map_domain_page(m3mfn);
@@ -1647,13 +1647,13 @@ sh_make_monitor_table(struct vcpu *v)
* If the necessary tables are not present in the guest, they return NULL. */
/* N.B. The use of GUEST_PAGING_LEVELS here is correct. If the shadow has
- * more levels than the guest, the upper levels are always fixed and do not
- * reflect any information from the guest, so we do not use these functions
+ * more levels than the guest, the upper levels are always fixed and do not
+ * reflect any information from the guest, so we do not use these functions
* to access them. */
#if GUEST_PAGING_LEVELS >= 4
-static shadow_l4e_t * shadow_get_and_create_l4e(struct vcpu *v,
- walk_t *gw,
+static shadow_l4e_t * shadow_get_and_create_l4e(struct vcpu *v,
+ walk_t *gw,
mfn_t *sl4mfn)
{
/* There is always a shadow of the top level table. Get it. */
@@ -1662,8 +1662,8 @@ static shadow_l4e_t * shadow_get_and_create_l4e(struct vcpu *v,
return sh_linear_l4_table(v) + shadow_l4_linear_offset(gw->va);
}
-static shadow_l3e_t * shadow_get_and_create_l3e(struct vcpu *v,
- walk_t *gw,
+static shadow_l3e_t * shadow_get_and_create_l3e(struct vcpu *v,
+ walk_t *gw,
mfn_t *sl3mfn,
fetch_type_t ft,
int *resync)
@@ -1674,18 +1674,18 @@ static shadow_l3e_t * shadow_get_and_create_l3e(struct vcpu *v,
/* Get the l4e */
sl4e = shadow_get_and_create_l4e(v, gw, &sl4mfn);
ASSERT(sl4e != NULL);
- if ( shadow_l4e_get_flags(*sl4e) & _PAGE_PRESENT )
+ if ( shadow_l4e_get_flags(*sl4e) & _PAGE_PRESENT )
{
*sl3mfn = shadow_l4e_get_mfn(*sl4e);
ASSERT(mfn_valid(*sl3mfn));
- }
- else
+ }
+ else
{
int r;
shadow_l4e_t new_sl4e;
/* No l3 shadow installed: find and install it. */
*sl3mfn = get_shadow_status(v, gw->l3mfn, SH_type_l3_shadow);
- if ( !mfn_valid(*sl3mfn) )
+ if ( !mfn_valid(*sl3mfn) )
{
/* No l3 shadow of this page exists at all: make one. */
*sl3mfn = sh_make_shadow(v, gw->l3mfn, SH_type_l3_shadow);
@@ -1708,8 +1708,8 @@ static shadow_l3e_t * shadow_get_and_create_l3e(struct vcpu *v,
#endif /* GUEST_PAGING_LEVELS >= 4 */
-static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
- walk_t *gw,
+static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
+ walk_t *gw,
mfn_t *sl2mfn,
fetch_type_t ft,
int *resync)
@@ -1720,13 +1720,13 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
if ( !mfn_valid(gw->l2mfn) ) return NULL; /* No guest page. */
/* Get the l3e */
sl3e = shadow_get_and_create_l3e(v, gw, &sl3mfn, ft, resync);
- if ( sl3e == NULL ) return NULL;
- if ( shadow_l3e_get_flags(*sl3e) & _PAGE_PRESENT )
+ if ( sl3e == NULL ) return NULL;
+ if ( shadow_l3e_get_flags(*sl3e) & _PAGE_PRESENT )
{
*sl2mfn = shadow_l3e_get_mfn(*sl3e);
ASSERT(mfn_valid(*sl2mfn));
- }
- else
+ }
+ else
{
int r;
shadow_l3e_t new_sl3e;
@@ -1740,7 +1740,7 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
/* No l2 shadow installed: find and install it. */
*sl2mfn = get_shadow_status(v, gw->l2mfn, t);
- if ( !mfn_valid(*sl2mfn) )
+ if ( !mfn_valid(*sl2mfn) )
{
/* No l2 shadow of this page exists at all: make one. */
*sl2mfn = sh_make_shadow(v, gw->l2mfn, t);
@@ -1750,7 +1750,7 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
r = shadow_set_l3e(v, sl3e, new_sl3e, sl3mfn);
ASSERT((r & SHADOW_SET_FLUSH) == 0);
if ( r & SHADOW_SET_ERROR )
- return NULL;
+ return NULL;
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC )
*resync |= 1;
@@ -1762,9 +1762,9 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
#elif GUEST_PAGING_LEVELS == 3 /* PAE... */
/* We never demand-shadow PAE l3es: they are only created in
* sh_update_cr3(). Check if the relevant sl3e is present. */
- shadow_l3e_t *sl3e = ((shadow_l3e_t *)&v->arch.paging.shadow.l3table)
+ shadow_l3e_t *sl3e = ((shadow_l3e_t *)&v->arch.paging.shadow.l3table)
+ shadow_l3_linear_offset(gw->va);
- if ( !(shadow_l3e_get_flags(*sl3e) & _PAGE_PRESENT) )
+ if ( !(shadow_l3e_get_flags(*sl3e) & _PAGE_PRESENT) )
return NULL;
*sl2mfn = shadow_l3e_get_mfn(*sl3e);
ASSERT(mfn_valid(*sl2mfn));
@@ -1778,12 +1778,12 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
(void) shadow_l2_index(sl2mfn, guest_l2_table_offset(gw->va));
/* Reading the top level table is always valid. */
return sh_linear_l2_table(v) + shadow_l2_linear_offset(gw->va);
-#endif
+#endif
}
-static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
- walk_t *gw,
+static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
+ walk_t *gw,
mfn_t *sl1mfn,
fetch_type_t ft)
{
@@ -1797,38 +1797,38 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
/* Install the sl1 in the l2e if it wasn't there or if we need to
* re-do it to fix a PSE dirty bit. */
- if ( shadow_l2e_get_flags(*sl2e) & _PAGE_PRESENT
+ if ( shadow_l2e_get_flags(*sl2e) & _PAGE_PRESENT
&& likely(ft != ft_demand_write
- || (shadow_l2e_get_flags(*sl2e) & _PAGE_RW)
+ || (shadow_l2e_get_flags(*sl2e) & _PAGE_RW)
|| !(guest_l2e_get_flags(gw->l2e) & _PAGE_PSE)) )
{
*sl1mfn = shadow_l2e_get_mfn(*sl2e);
ASSERT(mfn_valid(*sl1mfn));
- }
- else
+ }
+ else
{
shadow_l2e_t new_sl2e;
int r, flags = guest_l2e_get_flags(gw->l2e);
/* No l1 shadow installed: find and install it. */
if ( !(flags & _PAGE_PRESENT) )
return NULL; /* No guest page. */
- if ( guest_supports_superpages(v) && (flags & _PAGE_PSE) )
+ if ( guest_supports_superpages(v) && (flags & _PAGE_PSE) )
{
/* Splintering a superpage */
gfn_t l2gfn = guest_l2e_get_gfn(gw->l2e);
*sl1mfn = get_fl1_shadow_status(v, l2gfn);
- if ( !mfn_valid(*sl1mfn) )
+ if ( !mfn_valid(*sl1mfn) )
{
/* No fl1 shadow of this superpage exists at all: make one. */
*sl1mfn = make_fl1_shadow(v, l2gfn);
}
- }
- else
+ }
+ else
{
/* Shadowing an actual guest l1 table */
if ( !mfn_valid(gw->l1mfn) ) return NULL; /* No guest page. */
*sl1mfn = get_shadow_status(v, gw->l1mfn, SH_type_l1_shadow);
- if ( !mfn_valid(*sl1mfn) )
+ if ( !mfn_valid(*sl1mfn) )
{
/* No l1 shadow of this page exists at all: make one. */
*sl1mfn = sh_make_shadow(v, gw->l1mfn, SH_type_l1_shadow);
@@ -1837,7 +1837,7 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
/* Install the new sl1 table in the sl2e */
l2e_propagate_from_guest(v, gw->l2e, *sl1mfn, &new_sl2e, ft);
r = shadow_set_l2e(v, sl2e, new_sl2e, sl2mfn);
- ASSERT((r & SHADOW_SET_FLUSH) == 0);
+ ASSERT((r & SHADOW_SET_FLUSH) == 0);
if ( r & SHADOW_SET_ERROR )
return NULL;
@@ -1863,7 +1863,7 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
/**************************************************************************/
-/* Destructors for shadow tables:
+/* Destructors for shadow tables:
* Unregister the shadow, decrement refcounts of any entries present in it,
* and release the memory.
*
@@ -1890,16 +1890,16 @@ void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
delete_shadow_status(v, gmfn, t, smfn);
shadow_demote(v, gmfn, t);
/* Decrement refcounts of all the old entries */
- sl4mfn = smfn;
+ sl4mfn = smfn;
SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, v->domain, {
- if ( shadow_l4e_get_flags(*sl4e) & _PAGE_PRESENT )
+ if ( shadow_l4e_get_flags(*sl4e) & _PAGE_PRESENT )
{
sh_put_ref(v, shadow_l4e_get_mfn(*sl4e),
- (((paddr_t)mfn_x(sl4mfn)) << PAGE_SHIFT)
+ (((paddr_t)mfn_x(sl4mfn)) << PAGE_SHIFT)
| ((unsigned long)sl4e & ~PAGE_MASK));
}
});
-
+
/* Put the memory back in the pool */
shadow_free(v->domain, smfn);
}
@@ -1922,11 +1922,11 @@ void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
shadow_demote(v, gmfn, t);
/* Decrement refcounts of all the old entries */
- sl3mfn = smfn;
+ sl3mfn = smfn;
SHADOW_FOREACH_L3E(sl3mfn, sl3e, 0, 0, {
- if ( shadow_l3e_get_flags(*sl3e) & _PAGE_PRESENT )
+ if ( shadow_l3e_get_flags(*sl3e) & _PAGE_PRESENT )
sh_put_ref(v, shadow_l3e_get_mfn(*sl3e),
- (((paddr_t)mfn_x(sl3mfn)) << PAGE_SHIFT)
+ (((paddr_t)mfn_x(sl3mfn)) << PAGE_SHIFT)
| ((unsigned long)sl3e & ~PAGE_MASK));
});
@@ -1961,9 +1961,9 @@ void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
/* Decrement refcounts of all the old entries */
sl2mfn = smfn;
SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, v->domain, {
- if ( shadow_l2e_get_flags(*sl2e) & _PAGE_PRESENT )
+ if ( shadow_l2e_get_flags(*sl2e) & _PAGE_PRESENT )
sh_put_ref(v, shadow_l2e_get_mfn(*sl2e),
- (((paddr_t)mfn_x(sl2mfn)) << PAGE_SHIFT)
+ (((paddr_t)mfn_x(sl2mfn)) << PAGE_SHIFT)
| ((unsigned long)sl2e & ~PAGE_MASK));
});
@@ -1989,17 +1989,17 @@ void sh_destroy_l1_shadow(struct vcpu *v, mfn_t smfn)
gfn_t gfn = _gfn(sp->v.sh.back);
delete_fl1_shadow_status(v, gfn, smfn);
}
- else
+ else
{
mfn_t gmfn = backpointer(sp);
delete_shadow_status(v, gmfn, t, smfn);
shadow_demote(v, gmfn, t);
}
-
+
if ( shadow_mode_refcounts(d) )
{
/* Decrement refcounts of all the old entries */
- mfn_t sl1mfn = smfn;
+ mfn_t sl1mfn = smfn;
SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, 0, {
if ( (shadow_l1e_get_flags(*sl1e) & _PAGE_PRESENT)
&& !sh_l1e_is_magic(*sl1e) ) {
@@ -2008,7 +2008,7 @@ void sh_destroy_l1_shadow(struct vcpu *v, mfn_t smfn)
}
});
}
-
+
/* Put the memory back in the pool */
shadow_free(v->domain, smfn);
}
@@ -2025,8 +2025,8 @@ void sh_destroy_monitor_table(struct vcpu *v, mfn_t mmfn)
l4_pgentry_t *l4e = sh_map_domain_page(mmfn);
l3_pgentry_t *l3e;
int linear_slot = shadow_l4_table_offset(SH_LINEAR_PT_VIRT_START);
-
- /* Need to destroy the l3 and l2 monitor pages used
+
+ /* Need to destroy the l3 and l2 monitor pages used
* for the linear map */
ASSERT(l4e_get_flags(l4e[linear_slot]) & _PAGE_PRESENT);
m3mfn = _mfn(l4e_get_pfn(l4e[linear_slot]));
@@ -2060,18 +2060,18 @@ void sh_destroy_monitor_table(struct vcpu *v, mfn_t mmfn)
/**************************************************************************/
/* Functions to destroy non-Xen mappings in a pagetable hierarchy.
* These are called from common code when we are running out of shadow
- * memory, and unpinning all the top-level shadows hasn't worked.
+ * memory, and unpinning all the top-level shadows hasn't worked.
*
* With user_only == 1, we leave guest kernel-mode mappings in place too,
* unhooking only the user-mode mappings
*
- * This implementation is pretty crude and slow, but we hope that it won't
+ * This implementation is pretty crude and slow, but we hope that it won't
* be called very often. */
#if GUEST_PAGING_LEVELS == 2
void sh_unhook_32b_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
-{
+{
shadow_l2e_t *sl2e;
SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, v->domain, {
if ( !user_only || (sl2e->l2 & _PAGE_USER) )
@@ -2109,7 +2109,7 @@ void sh_unhook_64b_mappings(struct vcpu *v, mfn_t sl4mfn, int user_only)
* These functions require a pointer to the shadow entry that will be updated.
*/
-/* These functions take a new guest entry, translate it to shadow and write
+/* These functions take a new guest entry, translate it to shadow and write
* the shadow entry.
*
* They return the same bitmaps as the shadow_set_lXe() functions.
@@ -2240,7 +2240,7 @@ static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
mfn_t gl1mfn = get_gfn_query_unlocked(v->domain, gfn_x(gl1gfn),
&p2mt);
if ( p2m_is_ram(p2mt) )
- sl1mfn = get_shadow_status(v, gl1mfn, SH_type_l1_shadow);
+ sl1mfn = get_shadow_status(v, gl1mfn, SH_type_l1_shadow);
else if ( p2mt != p2m_populate_on_demand )
result |= SHADOW_SET_ERROR;
}
@@ -2275,7 +2275,7 @@ static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
gl1mfn = backpointer(mfn_to_page(sl1mfn));
- if ( mfn_valid(gl1mfn)
+ if ( mfn_valid(gl1mfn)
&& mfn_is_out_of_sync(gl1mfn) )
{
/* Update the OOS snapshot. */
@@ -2295,7 +2295,7 @@ static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/**************************************************************************/
-/* Special validation function for re-syncing out-of-sync shadows.
+/* Special validation function for re-syncing out-of-sync shadows.
* Walks the *shadow* page, and for every entry that it finds,
* revalidates the guest entry that corresponds to it.
* N.B. This function is called with the vcpu that unsynced the page,
@@ -2342,10 +2342,10 @@ void sh_resync_l1(struct vcpu *v, mfn_t gl1mfn, mfn_t snpmfn)
ASSERT(!(rc & SHADOW_SET_FLUSH));
}
-/* Figure out whether it's definitely safe not to sync this l1 table.
- * That is: if we can tell that it's only used once, and that the
- * toplevel shadow responsible is not one of ours.
- * N.B. This function is called with the vcpu that required the resync,
+/* Figure out whether it's definitely safe not to sync this l1 table.
+ * That is: if we can tell that it's only used once, and that the
+ * toplevel shadow responsible is not one of ours.
+ * N.B. This function is called with the vcpu that required the resync,
* *not* the one that originally unsynced the page, but it is
* called in the *mode* of the vcpu that unsynced it. Clear? Good. */
int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
@@ -2366,7 +2366,7 @@ int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
smfn = _mfn(sp->up >> PAGE_SHIFT);
ASSERT(mfn_valid(smfn));
-#if (SHADOW_PAGING_LEVELS == 4)
+#if (SHADOW_PAGING_LEVELS == 4)
/* up to l3 */
sp = mfn_to_page(smfn);
ASSERT(sh_type_has_up_pointer(v, SH_type_l2_shadow));
@@ -2385,15 +2385,15 @@ int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
#endif
if ( pagetable_get_pfn(v->arch.shadow_table[0]) == mfn_x(smfn)
-#if (SHADOW_PAGING_LEVELS == 3)
+#if (SHADOW_PAGING_LEVELS == 3)
|| pagetable_get_pfn(v->arch.shadow_table[1]) == mfn_x(smfn)
|| pagetable_get_pfn(v->arch.shadow_table[2]) == mfn_x(smfn)
- || pagetable_get_pfn(v->arch.shadow_table[3]) == mfn_x(smfn)
+ || pagetable_get_pfn(v->arch.shadow_table[3]) == mfn_x(smfn)
#endif
)
return 0;
-
- /* Only in use in one toplevel shadow, and it's not the one we're
+
+ /* Only in use in one toplevel shadow, and it's not the one we're
* running on */
return 1;
}
@@ -2401,15 +2401,15 @@ int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
/**************************************************************************/
-/* Functions which translate and install the shadows of arbitrary guest
+/* Functions which translate and install the shadows of arbitrary guest
* entries that we have just seen the guest write. */
-static inline int
+static inline int
sh_map_and_validate(struct vcpu *v, mfn_t gmfn,
- void *new_gp, u32 size, u32 sh_type,
+ void *new_gp, u32 size, u32 sh_type,
u32 (*shadow_index)(mfn_t *smfn, u32 idx),
- int (*validate_ge)(struct vcpu *v, void *ge,
+ int (*validate_ge)(struct vcpu *v, void *ge,
mfn_t smfn, void *se))
/* Generic function for mapping and validating. */
{
@@ -2462,25 +2462,25 @@ sh_map_and_validate_gl4e(struct vcpu *v, mfn_t gl4mfn,
void *new_gl4p, u32 size)
{
#if GUEST_PAGING_LEVELS >= 4
- return sh_map_and_validate(v, gl4mfn, new_gl4p, size,
- SH_type_l4_shadow,
- shadow_l4_index,
+ return sh_map_and_validate(v, gl4mfn, new_gl4p, size,
+ SH_type_l4_shadow,
+ shadow_l4_index,
validate_gl4e);
#else // ! GUEST_PAGING_LEVELS >= 4
SHADOW_ERROR("called in wrong paging mode!\n");
BUG();
return 0;
-#endif
+#endif
}
-
+
int
sh_map_and_validate_gl3e(struct vcpu *v, mfn_t gl3mfn,
void *new_gl3p, u32 size)
{
#if GUEST_PAGING_LEVELS >= 4
- return sh_map_and_validate(v, gl3mfn, new_gl3p, size,
- SH_type_l3_shadow,
- shadow_l3_index,
+ return sh_map_and_validate(v, gl3mfn, new_gl3p, size,
+ SH_type_l3_shadow,
+ shadow_l3_index,
validate_gl3e);
#else // ! GUEST_PAGING_LEVELS >= 4
SHADOW_ERROR("called in wrong paging mode!\n");
@@ -2493,9 +2493,9 @@ int
sh_map_and_validate_gl2e(struct vcpu *v, mfn_t gl2mfn,
void *new_gl2p, u32 size)
{
- return sh_map_and_validate(v, gl2mfn, new_gl2p, size,
- SH_type_l2_shadow,
- shadow_l2_index,
+ return sh_map_and_validate(v, gl2mfn, new_gl2p, size,
+ SH_type_l2_shadow,
+ shadow_l2_index,
validate_gl2e);
}
@@ -2504,9 +2504,9 @@ sh_map_and_validate_gl2he(struct vcpu *v, mfn_t gl2mfn,
void *new_gl2p, u32 size)
{
#if GUEST_PAGING_LEVELS >= 3
- return sh_map_and_validate(v, gl2mfn, new_gl2p, size,
- SH_type_l2h_shadow,
- shadow_l2_index,
+ return sh_map_and_validate(v, gl2mfn, new_gl2p, size,
+ SH_type_l2h_shadow,
+ shadow_l2_index,
validate_gl2e);
#else /* Non-PAE guests don't have different kinds of l2 table */
SHADOW_ERROR("called in wrong paging mode!\n");
@@ -2519,9 +2519,9 @@ int
sh_map_and_validate_gl1e(struct vcpu *v, mfn_t gl1mfn,
void *new_gl1p, u32 size)
{
- return sh_map_and_validate(v, gl1mfn, new_gl1p, size,
- SH_type_l1_shadow,
- shadow_l1_index,
+ return sh_map_and_validate(v, gl1mfn, new_gl1p, size,
+ SH_type_l1_shadow,
+ shadow_l1_index,
validate_gl1e);
}
@@ -2572,7 +2572,7 @@ static inline void reset_early_unshadow(struct vcpu *v)
/**************************************************************************/
-/* Optimization: Prefetch multiple L1 entries. This is called after we have
+/* Optimization: Prefetch multiple L1 entries. This is called after we have
* demand-faulted a shadow l1e in the fault handler, to see if it's
* worth fetching some more.
*/
@@ -2582,7 +2582,7 @@ static inline void reset_early_unshadow(struct vcpu *v)
/* XXX magic number */
#define PREFETCH_DISTANCE 32
-static void sh_prefetch(struct vcpu *v, walk_t *gw,
+static void sh_prefetch(struct vcpu *v, walk_t *gw,
shadow_l1e_t *ptr_sl1e, mfn_t sl1mfn)
{
int i, dist;
@@ -2621,7 +2621,7 @@ static void sh_prefetch(struct vcpu *v, walk_t *gw,
#endif /* OOS */
}
- for ( i = 1; i < dist ; i++ )
+ for ( i = 1; i < dist ; i++ )
{
/* No point in prefetching if there's already a shadow */
if ( ptr_sl1e[i].l1 != 0 )
@@ -2634,18 +2634,18 @@ static void sh_prefetch(struct vcpu *v, walk_t *gw,
/* Not worth continuing if we hit an entry that will need another
* fault for A/D-bit propagation anyway */
gflags = guest_l1e_get_flags(gl1e);
- if ( (gflags & _PAGE_PRESENT)
+ if ( (gflags & _PAGE_PRESENT)
&& (!(gflags & _PAGE_ACCESSED)
|| ((gflags & _PAGE_RW) && !(gflags & _PAGE_DIRTY))) )
break;
- }
- else
+ }
+ else
{
/* Fragmented superpage, unless we've been called wrongly */
ASSERT(guest_l2e_get_flags(gw->l2e) & _PAGE_PSE);
/* Increment the l1e's GFN by the right number of guest pages */
gl1e = guest_l1e_from_gfn(
- _gfn(gfn_x(guest_l1e_get_gfn(gw->l1e)) + i),
+ _gfn(gfn_x(guest_l1e_get_gfn(gw->l1e)) + i),
guest_l1e_get_flags(gw->l1e));
}
@@ -2715,7 +2715,7 @@ static inline void trace_shadow_fixup(guest_l1e_t gl1e,
__trace_var(event, 0/*!tsc*/, sizeof(d), &d);
}
}
-
+
static inline void trace_not_shadow_fault(guest_l1e_t gl1e,
guest_va_t va)
{
@@ -2739,7 +2739,7 @@ static inline void trace_not_shadow_fault(guest_l1e_t gl1e,
__trace_var(event, 0/*!tsc*/, sizeof(d), &d);
}
}
-
+
static inline void trace_shadow_emulate_other(u32 event,
guest_va_t va,
gfn_t gfn)
@@ -2807,8 +2807,8 @@ static inline void trace_shadow_emulate(guest_l1e_t gl1e, unsigned long va)
* shadow code (and the guest should retry) or 0 if it is not (and the
* fault should be handled elsewhere or passed to the guest). */
-static int sh_page_fault(struct vcpu *v,
- unsigned long va,
+static int sh_page_fault(struct vcpu *v,
+ unsigned long va,
struct cpu_user_regs *regs)
{
struct domain *d = v->domain;
@@ -2848,7 +2848,7 @@ static int sh_page_fault(struct vcpu *v,
* Then try to emulate early to avoid lock aquisition.
*/
if ( v->arch.paging.last_write_emul_ok
- && v->arch.paging.shadow.last_emulated_frame == (va >> PAGE_SHIFT) )
+ && v->arch.paging.shadow.last_emulated_frame == (va >> PAGE_SHIFT) )
{
/* check whether error code is 3, or else fall back to normal path
* in case of some validation is required
@@ -2858,7 +2858,7 @@ static int sh_page_fault(struct vcpu *v,
fast_emul = 1;
gmfn = _mfn(v->arch.paging.shadow.last_emulated_mfn);
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Fall back to the slow path if we're trying to emulate
writes to an out of sync page. */
if ( mfn_valid(gmfn) && mfn_is_out_of_sync(gmfn) )
@@ -2886,7 +2886,7 @@ static int sh_page_fault(struct vcpu *v,
#if (SHADOW_OPTIMIZATIONS & SHOPT_FAST_FAULT_PATH)
if ( (regs->error_code & PFEC_reserved_bit) )
{
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* First, need to check that this isn't an out-of-sync
* shadow l1e. If it is, we fall back to the slow path, which
* will sync it up again. */
@@ -2902,7 +2902,7 @@ static int sh_page_fault(struct vcpu *v,
shadow_l2e_get_mfn(sl2e))))
|| unlikely(mfn_is_out_of_sync(gl1mfn)) )
{
- /* Hit the slow path as if there had been no
+ /* Hit the slow path as if there had been no
* shadow entry at all, and let it tidy up */
ASSERT(regs->error_code & PFEC_page_present);
regs->error_code ^= (PFEC_reserved_bit|PFEC_page_present);
@@ -2910,10 +2910,10 @@ static int sh_page_fault(struct vcpu *v,
}
}
#endif /* SHOPT_OUT_OF_SYNC */
- /* The only reasons for reserved bits to be set in shadow entries
+ /* The only reasons for reserved bits to be set in shadow entries
* are the two "magic" shadow_l1e entries. */
- if ( likely((__copy_from_user(&sl1e,
- (sh_linear_l1_table(v)
+ if ( likely((__copy_from_user(&sl1e,
+ (sh_linear_l1_table(v)
+ shadow_l1_linear_offset(va)),
sizeof(sl1e)) == 0)
&& sh_l1e_is_magic(sl1e)) )
@@ -2935,8 +2935,8 @@ static int sh_page_fault(struct vcpu *v,
{
/* Magic MMIO marker: extract gfn for MMIO address */
ASSERT(sh_l1e_is_mmio(sl1e));
- gpa = (((paddr_t)(gfn_x(sh_l1e_mmio_get_gfn(sl1e))))
- << PAGE_SHIFT)
+ gpa = (((paddr_t)(gfn_x(sh_l1e_mmio_get_gfn(sl1e))))
+ << PAGE_SHIFT)
| (va & ~PAGE_MASK);
}
perfc_incr(shadow_fault_fast_mmio);
@@ -2949,24 +2949,24 @@ static int sh_page_fault(struct vcpu *v,
else
{
/* This should be exceptionally rare: another vcpu has fixed
- * the tables between the fault and our reading the l1e.
+ * the tables between the fault and our reading the l1e.
* Retry and let the hardware give us the right fault next time. */
perfc_incr(shadow_fault_fast_fail);
- SHADOW_PRINTK("fast path false alarm!\n");
+ SHADOW_PRINTK("fast path false alarm!\n");
trace_shadow_gen(TRC_SHADOW_FALSE_FAST_PATH, va);
return EXCRET_fault_fixed;
}
}
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
page_fault_slow_path:
#endif
#endif /* SHOPT_FAST_FAULT_PATH */
/* Detect if this page fault happened while we were already in Xen
* doing a shadow operation. If that happens, the only thing we can
- * do is let Xen's normal fault handlers try to fix it. In any case,
- * a diagnostic trace of the fault will be more useful than
+ * do is let Xen's normal fault handlers try to fix it. In any case,
+ * a diagnostic trace of the fault will be more useful than
* a BUG() when we try to take the lock again. */
if ( unlikely(paging_locked_by_me(d)) )
{
@@ -2980,7 +2980,7 @@ static int sh_page_fault(struct vcpu *v,
/* The walk is done in a lock-free style, with some sanity check
* postponed after grabbing paging lock later. Those delayed checks
* will make sure no inconsistent mapping being translated into
- * shadow page table. */
+ * shadow page table. */
version = atomic_read(&d->arch.paging.shadow.gtable_dirty_version);
rmb();
rc = sh_walk_guest_tables(v, va, &gw, regs->error_code);
@@ -3001,9 +3001,9 @@ static int sh_page_fault(struct vcpu *v,
goto propagate;
}
- /* It's possible that the guest has put pagetables in memory that it has
+ /* It's possible that the guest has put pagetables in memory that it has
* already used for some special purpose (ioreq pages, or granted pages).
- * If that happens we'll have killed the guest already but it's still not
+ * If that happens we'll have killed the guest already but it's still not
* safe to propagate entries out of the guest PT so get out now. */
if ( unlikely(d->is_shutting_down && d->shutdown_code == SHUTDOWN_crash) )
{
@@ -3019,12 +3019,12 @@ static int sh_page_fault(struct vcpu *v,
gfn = guest_l1e_get_gfn(gw.l1e);
gmfn = get_gfn(d, gfn, &p2mt);
- if ( shadow_mode_refcounts(d) &&
+ if ( shadow_mode_refcounts(d) &&
((!p2m_is_valid(p2mt) && !p2m_is_grant(p2mt)) ||
(!p2m_is_mmio(p2mt) && !mfn_valid(gmfn))) )
{
perfc_incr(shadow_fault_bail_bad_gfn);
- SHADOW_PRINTK("BAD gfn=%"SH_PRI_gfn" gmfn=%"PRI_mfn"\n",
+ SHADOW_PRINTK("BAD gfn=%"SH_PRI_gfn" gmfn=%"PRI_mfn"\n",
gfn_x(gfn), mfn_x(gmfn));
reset_early_unshadow(v);
put_gfn(d, gfn_x(gfn));
@@ -3033,7 +3033,7 @@ static int sh_page_fault(struct vcpu *v,
#if (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB)
/* Remember this successful VA->GFN translation for later. */
- vtlb_insert(v, va >> PAGE_SHIFT, gfn_x(gfn),
+ vtlb_insert(v, va >> PAGE_SHIFT, gfn_x(gfn),
regs->error_code | PFEC_page_present);
#endif /* (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB) */
@@ -3053,7 +3053,7 @@ static int sh_page_fault(struct vcpu *v,
shadow_prealloc(d,
SH_type_l1_shadow,
GUEST_PAGING_LEVELS < 4 ? 1 : GUEST_PAGING_LEVELS - 1);
-
+
rc = gw_remove_write_accesses(v, va, &gw);
/* First bit set: Removed write access to a page. */
@@ -3088,10 +3088,10 @@ static int sh_page_fault(struct vcpu *v,
shadow_audit_tables(v);
sh_audit_gw(v, &gw);
- /* Acquire the shadow. This must happen before we figure out the rights
+ /* Acquire the shadow. This must happen before we figure out the rights
* for the shadow entry, since we might promote a page here. */
ptr_sl1e = shadow_get_and_create_l1e(v, &gw, &sl1mfn, ft);
- if ( unlikely(ptr_sl1e == NULL) )
+ if ( unlikely(ptr_sl1e == NULL) )
{
/* Couldn't get the sl1e! Since we know the guest entries
* are OK, this can only have been caused by a failed
@@ -3146,15 +3146,15 @@ static int sh_page_fault(struct vcpu *v,
r = shadow_set_l1e(v, ptr_sl1e, sl1e, p2mt, sl1mfn);
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
- if ( mfn_valid(gw.l1mfn)
+ if ( mfn_valid(gw.l1mfn)
&& mfn_is_out_of_sync(gw.l1mfn) )
{
/* Update the OOS snapshot. */
mfn_t snpmfn = oos_snapshot_lookup(v, gw.l1mfn);
guest_l1e_t *snp;
-
+
ASSERT(mfn_valid(snpmfn));
-
+
snp = sh_map_domain_page(snpmfn);
snp[guest_l1_table_offset(va)] = gw.l1e;
sh_unmap_domain_page(snp);
@@ -3168,7 +3168,7 @@ static int sh_page_fault(struct vcpu *v,
/* Need to emulate accesses to page tables */
if ( sh_mfn_is_a_page_table(gmfn)
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Unless they've been allowed to go out of sync with their
shadows and we don't need to unshadow it. */
&& !(mfn_is_out_of_sync(gmfn)
@@ -3202,10 +3202,10 @@ static int sh_page_fault(struct vcpu *v,
/* In HVM guests, we force CR0.WP always to be set, so that the
* pagetables are always write-protected. If the guest thinks
* CR0.WP is clear, we must emulate faulting supervisor writes to
- * allow the guest to write through read-only PTEs. Emulate if the
+ * allow the guest to write through read-only PTEs. Emulate if the
* fault was a non-user write to a present page. */
- if ( is_hvm_domain(d)
- && unlikely(!hvm_wp_enabled(v))
+ if ( is_hvm_domain(d)
+ && unlikely(!hvm_wp_enabled(v))
&& regs->error_code == (PFEC_write_access|PFEC_page_present)
&& mfn_valid(gmfn) )
{
@@ -3237,7 +3237,7 @@ static int sh_page_fault(struct vcpu *v,
*/
if ( (regs->error_code & PFEC_user_mode) )
{
- SHADOW_PRINTK("user-mode fault to PT, unshadowing mfn %#lx\n",
+ SHADOW_PRINTK("user-mode fault to PT, unshadowing mfn %#lx\n",
mfn_x(gmfn));
perfc_incr(shadow_fault_emulate_failed);
sh_remove_shadows(v, gmfn, 0 /* thorough */, 1 /* must succeed */);
@@ -3318,7 +3318,7 @@ static int sh_page_fault(struct vcpu *v,
}
#endif
gdprintk(XENLOG_DEBUG, "write to pagetable during event "
- "injection: cr2=%#lx, mfn=%#lx\n",
+ "injection: cr2=%#lx, mfn=%#lx\n",
va, mfn_x(gmfn));
sh_remove_shadows(v, gmfn, 0 /* thorough */, 1 /* must succeed */);
trace_shadow_emulate_other(TRC_SHADOW_EMULATE_UNSHADOW_EVTINJ,
@@ -3327,7 +3327,7 @@ static int sh_page_fault(struct vcpu *v,
}
}
- SHADOW_PRINTK("emulate: eip=%#lx esp=%#lx\n",
+ SHADOW_PRINTK("emulate: eip=%#lx esp=%#lx\n",
(unsigned long)regs->eip, (unsigned long)regs->esp);
emul_ops = shadow_init_emulation(&emul_ctxt, regs);
@@ -3349,10 +3349,10 @@ static int sh_page_fault(struct vcpu *v,
v->arch.paging.last_write_emul_ok = 0;
}
#endif
- SHADOW_PRINTK("emulator failure, unshadowing mfn %#lx\n",
+ SHADOW_PRINTK("emulator failure, unshadowing mfn %#lx\n",
mfn_x(gmfn));
- /* If this is actually a page table, then we have a bug, and need
- * to support more operations in the emulator. More likely,
+ /* If this is actually a page table, then we have a bug, and need
+ * to support more operations in the emulator. More likely,
* though, this is a hint that this page should not be shadowed. */
shadow_remove_all_shadows(v, gmfn);
@@ -3387,7 +3387,7 @@ static int sh_page_fault(struct vcpu *v,
if ( r == X86EMUL_OKAY ) {
int i, emulation_count=0;
this_cpu(trace_emulate_initial_va) = va;
- /* Emulate up to four extra instructions in the hope of catching
+ /* Emulate up to four extra instructions in the hope of catching
* the "second half" of a 64-bit pagetable write. */
for ( i = 0 ; i < 4 ; i++ )
{
@@ -3395,7 +3395,7 @@ static int sh_page_fault(struct vcpu *v,
v->arch.paging.last_write_was_pt = 0;
r = x86_emulate(&emul_ctxt.ctxt, emul_ops);
if ( r == X86EMUL_OKAY )
- {
+ {
emulation_count++;
if ( v->arch.paging.last_write_was_pt )
{
@@ -3403,7 +3403,7 @@ static int sh_page_fault(struct vcpu *v,
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_EMULATION_2ND_PT_WRITTEN);
break; /* Don't emulate past the other half of the write */
}
- else
+ else
perfc_incr(shadow_em_ex_non_pt);
}
else
@@ -3459,7 +3459,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
{
mfn_t sl1mfn;
shadow_l2e_t sl2e;
-
+
perfc_incr(shadow_invlpg);
#if (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB)
@@ -3472,7 +3472,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
#endif
/* First check that we can safely read the shadow l2e. SMP/PAE linux can
- * run as high as 6% of invlpg calls where we haven't shadowed the l2
+ * run as high as 6% of invlpg calls where we haven't shadowed the l2
* yet. */
#if SHADOW_PAGING_LEVELS == 4
{
@@ -3484,7 +3484,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
/* This must still be a copy-from-user because we don't have the
* paging lock, and the higher-level shadows might disappear
* under our feet. */
- if ( __copy_from_user(&sl3e, (sh_linear_l3_table(v)
+ if ( __copy_from_user(&sl3e, (sh_linear_l3_table(v)
+ shadow_l3_linear_offset(va)),
sizeof (sl3e)) != 0 )
{
@@ -3503,7 +3503,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
/* This must still be a copy-from-user because we don't have the shadow
* lock, and the higher-level shadows might disappear under our feet. */
- if ( __copy_from_user(&sl2e,
+ if ( __copy_from_user(&sl2e,
sh_linear_l2_table(v) + shadow_l2_linear_offset(va),
sizeof (sl2e)) != 0 )
{
@@ -3529,12 +3529,12 @@ sh_invlpg(struct vcpu *v, unsigned long va)
return 0;
}
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Check to see if the SL1 is out of sync. */
{
mfn_t gl1mfn = backpointer(mfn_to_page(sl1mfn));
struct page_info *pg = mfn_to_page(gl1mfn);
- if ( mfn_valid(gl1mfn)
+ if ( mfn_valid(gl1mfn)
&& page_is_out_of_sync(pg) )
{
/* The test above may give false positives, since we don't
@@ -3545,7 +3545,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
* have the paging lock last time we checked, and the
* higher-level shadows might have disappeared under our
* feet. */
- if ( __copy_from_user(&sl2e,
+ if ( __copy_from_user(&sl2e,
sh_linear_l2_table(v)
+ shadow_l2_linear_offset(va),
sizeof (sl2e)) != 0 )
@@ -3564,7 +3564,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
sl1mfn = shadow_l2e_get_mfn(sl2e);
gl1mfn = backpointer(mfn_to_page(sl1mfn));
pg = mfn_to_page(gl1mfn);
-
+
if ( likely(sh_mfn_is_a_page_table(gl1mfn)
&& page_is_out_of_sync(pg) ) )
{
@@ -3598,7 +3598,7 @@ sh_gva_to_gfn(struct vcpu *v, struct p2m_domain *p2m,
#if (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB)
/* Check the vTLB cache first */
unsigned long vtlb_gfn = vtlb_lookup(v, va, pfec[0]);
- if ( VALID_GFN(vtlb_gfn) )
+ if ( VALID_GFN(vtlb_gfn) )
return vtlb_gfn;
#endif /* (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB) */
@@ -3637,7 +3637,7 @@ sh_update_linear_entries(struct vcpu *v)
* is subtler. Normal linear mappings are made by having an entry
* in the top-level table that points to itself (shadow linear) or
* to the guest top-level table (guest linear). For PAE, to set up
- * a linear map requires us to copy the four top-level entries into
+ * a linear map requires us to copy the four top-level entries into
* level-2 entries. That means that every time we change a PAE l3e,
* we need to reflect the change into the copy.
*
@@ -3647,41 +3647,41 @@ sh_update_linear_entries(struct vcpu *v)
* For HVM guests, the linear pagetables are installed in the monitor
* tables (since we can't put them in the shadow). Shadow linear
* pagetables, which map the shadows, are at SH_LINEAR_PT_VIRT_START,
- * and we use the linear pagetable slot at LINEAR_PT_VIRT_START for
- * a linear pagetable of the monitor tables themselves. We have
+ * and we use the linear pagetable slot at LINEAR_PT_VIRT_START for
+ * a linear pagetable of the monitor tables themselves. We have
* the same issue of having to re-copy PAE l3 entries whevever we use
- * PAE shadows.
+ * PAE shadows.
*
- * Because HVM guests run on the same monitor tables regardless of the
- * shadow tables in use, the linear mapping of the shadow tables has to
- * be updated every time v->arch.shadow_table changes.
+ * Because HVM guests run on the same monitor tables regardless of the
+ * shadow tables in use, the linear mapping of the shadow tables has to
+ * be updated every time v->arch.shadow_table changes.
*/
/* Don't try to update the monitor table if it doesn't exist */
- if ( shadow_mode_external(d)
- && pagetable_get_pfn(v->arch.monitor_table) == 0 )
+ if ( shadow_mode_external(d)
+ && pagetable_get_pfn(v->arch.monitor_table) == 0 )
return;
#if SHADOW_PAGING_LEVELS == 4
-
+
/* For PV, one l4e points at the guest l4, one points at the shadow
- * l4. No maintenance required.
+ * l4. No maintenance required.
* For HVM, just need to update the l4e that points to the shadow l4. */
if ( shadow_mode_external(d) )
{
/* Use the linear map if we can; otherwise make a new mapping */
- if ( v == current )
+ if ( v == current )
{
- __linear_l4_table[l4_linear_offset(SH_LINEAR_PT_VIRT_START)] =
+ __linear_l4_table[l4_linear_offset(SH_LINEAR_PT_VIRT_START)] =
l4e_from_pfn(pagetable_get_pfn(v->arch.shadow_table[0]),
__PAGE_HYPERVISOR);
- }
+ }
else
- {
+ {
l4_pgentry_t *ml4e;
ml4e = sh_map_domain_page(pagetable_get_mfn(v->arch.monitor_table));
- ml4e[l4_table_offset(SH_LINEAR_PT_VIRT_START)] =
+ ml4e[l4_table_offset(SH_LINEAR_PT_VIRT_START)] =
l4e_from_pfn(pagetable_get_pfn(v->arch.shadow_table[0]),
__PAGE_HYPERVISOR);
sh_unmap_domain_page(ml4e);
@@ -3712,8 +3712,8 @@ sh_update_linear_entries(struct vcpu *v)
if ( v == current )
ml2e = __linear_l2_table
+ l2_linear_offset(SH_LINEAR_PT_VIRT_START);
- else
- {
+ else
+ {
mfn_t l3mfn, l2mfn;
l4_pgentry_t *ml4e;
l3_pgentry_t *ml3e;
@@ -3736,14 +3736,14 @@ sh_update_linear_entries(struct vcpu *v)
for ( i = 0; i < SHADOW_L3_PAGETABLE_ENTRIES; i++ )
{
- ml2e[i] =
- (shadow_l3e_get_flags(sl3e[i]) & _PAGE_PRESENT)
+ ml2e[i] =
+ (shadow_l3e_get_flags(sl3e[i]) & _PAGE_PRESENT)
? l2e_from_pfn(mfn_x(shadow_l3e_get_mfn(sl3e[i])),
- __PAGE_HYPERVISOR)
+ __PAGE_HYPERVISOR)
: l2e_empty();
}
- if ( v != current )
+ if ( v != current )
sh_unmap_domain_page(ml2e);
}
else
@@ -3758,11 +3758,11 @@ sh_update_linear_entries(struct vcpu *v)
/*
* Having modified the linear pagetable mapping, flush local host TLBs.
* This was not needed when vmenter/vmexit always had the side effect
- * of flushing host TLBs but, with ASIDs, it is possible to finish
- * this CR3 update, vmenter the guest, vmexit due to a page fault,
- * without an intervening host TLB flush. Then the page fault code
- * could use the linear pagetable to read a top-level shadow page
- * table entry. But, without this change, it would fetch the wrong
+ * of flushing host TLBs but, with ASIDs, it is possible to finish
+ * this CR3 update, vmenter the guest, vmexit due to a page fault,
+ * without an intervening host TLB flush. Then the page fault code
+ * could use the linear pagetable to read a top-level shadow page
+ * table entry. But, without this change, it would fetch the wrong
* value due to a stale TLB.
*/
flush_tlb_local();
@@ -3815,16 +3815,16 @@ sh_detach_old_tables(struct vcpu *v)
/* Set up the top-level shadow and install it in slot 'slot' of shadow_table */
static void
-sh_set_toplevel_shadow(struct vcpu *v,
+sh_set_toplevel_shadow(struct vcpu *v,
int slot,
- mfn_t gmfn,
- unsigned int root_type)
+ mfn_t gmfn,
+ unsigned int root_type)
{
mfn_t smfn;
pagetable_t old_entry, new_entry;
struct domain *d = v->domain;
-
+
/* Remember the old contents of this slot */
old_entry = v->arch.shadow_table[slot];
@@ -3845,7 +3845,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
smfn = sh_make_shadow(v, gmfn, root_type);
}
ASSERT(mfn_valid(smfn));
-
+
/* Pin the shadow and put it (back) on the list of pinned shadows */
if ( sh_pin(v, smfn) == 0 )
{
@@ -3892,10 +3892,10 @@ sh_update_cr3(struct vcpu *v, int do_locking)
* Paravirtual guests should set v->arch.guest_table (and guest_table_user,
* if appropriate).
* HVM guests should also make sure hvm_get_guest_cntl_reg(v, 3) works;
- * this function will call hvm_update_guest_cr(v, 3) to tell them where the
+ * this function will call hvm_update_guest_cr(v, 3) to tell them where the
* shadow tables are.
- * If do_locking != 0, assume we are being called from outside the
- * shadow code, and must take and release the paging lock; otherwise
+ * If do_locking != 0, assume we are being called from outside the
+ * shadow code, and must take and release the paging lock; otherwise
* that is the caller's responsibility.
*/
{
@@ -3929,22 +3929,22 @@ sh_update_cr3(struct vcpu *v, int do_locking)
////
//// vcpu->arch.guest_table is already set
////
-
-#ifndef NDEBUG
+
+#ifndef NDEBUG
/* Double-check that the HVM code has sent us a sane guest_table */
if ( is_hvm_domain(d) )
{
ASSERT(shadow_mode_external(d));
if ( hvm_paging_enabled(v) )
ASSERT(pagetable_get_pfn(v->arch.guest_table));
- else
+ else
ASSERT(v->arch.guest_table.pfn
== d->arch.paging.shadow.unpaged_pagetable.pfn);
}
#endif
SHADOW_PRINTK("d=%u v=%u guest_table=%05lx\n",
- d->domain_id, v->vcpu_id,
+ d->domain_id, v->vcpu_id,
(unsigned long)pagetable_get_pfn(v->arch.guest_table));
#if GUEST_PAGING_LEVELS == 4
@@ -3975,18 +3975,18 @@ sh_update_cr3(struct vcpu *v, int do_locking)
* table. We cache the current state of that table and shadow that,
* until the next CR3 write makes us refresh our cache. */
ASSERT(v->arch.paging.shadow.guest_vtable == NULL);
-
- if ( shadow_mode_external(d) )
+
+ if ( shadow_mode_external(d) )
/* Find where in the page the l3 table is */
guest_idx = guest_index((void *)v->arch.hvm_vcpu.guest_cr[3]);
else
- /* PV guest: l3 is at the start of a page */
- guest_idx = 0;
+ /* PV guest: l3 is at the start of a page */
+ guest_idx = 0;
// Ignore the low 2 bits of guest_idx -- they are really just
// cache control.
guest_idx &= ~3;
-
+
gl3e = ((guest_l3e_t *)sh_map_domain_page(gmfn)) + guest_idx;
for ( i = 0; i < 4 ; i++ )
v->arch.paging.shadow.gl3e[i] = gl3e[i];
@@ -4013,14 +4013,14 @@ sh_update_cr3(struct vcpu *v, int do_locking)
////
/* We revoke write access to the new guest toplevel page(s) before we
- * replace the old shadow pagetable(s), so that we can safely use the
+ * replace the old shadow pagetable(s), so that we can safely use the
* (old) shadow linear maps in the writeable mapping heuristics. */
#if GUEST_PAGING_LEVELS == 2
if ( sh_remove_write_access(v, gmfn, 2, 0) != 0 )
flush_tlb_mask(d->domain_dirty_cpumask);
sh_set_toplevel_shadow(v, 0, gmfn, SH_type_l2_shadow);
#elif GUEST_PAGING_LEVELS == 3
- /* PAE guests have four shadow_table entries, based on the
+ /* PAE guests have four shadow_table entries, based on the
* current values of the guest's four l3es. */
{
int flush = 0;
@@ -4039,24 +4039,24 @@ sh_update_cr3(struct vcpu *v, int do_locking)
flush |= sh_remove_write_access(v, gl2mfn, 2, 0);
}
}
- if ( flush )
+ if ( flush )
flush_tlb_mask(d->domain_dirty_cpumask);
/* Now install the new shadows. */
- for ( i = 0; i < 4; i++ )
+ for ( i = 0; i < 4; i++ )
{
if ( guest_l3e_get_flags(gl3e[i]) & _PAGE_PRESENT )
{
gl2gfn = guest_l3e_get_gfn(gl3e[i]);
gl2mfn = get_gfn_query_unlocked(d, gfn_x(gl2gfn), &p2mt);
if ( p2m_is_ram(p2mt) )
- sh_set_toplevel_shadow(v, i, gl2mfn, (i == 3)
- ? SH_type_l2h_shadow
+ sh_set_toplevel_shadow(v, i, gl2mfn, (i == 3)
+ ? SH_type_l2h_shadow
: SH_type_l2_shadow);
else
- sh_set_toplevel_shadow(v, i, _mfn(INVALID_MFN), 0);
+ sh_set_toplevel_shadow(v, i, _mfn(INVALID_MFN), 0);
}
else
- sh_set_toplevel_shadow(v, i, _mfn(INVALID_MFN), 0);
+ sh_set_toplevel_shadow(v, i, _mfn(INVALID_MFN), 0);
}
}
#elif GUEST_PAGING_LEVELS == 4
@@ -4064,11 +4064,11 @@ sh_update_cr3(struct vcpu *v, int do_locking)
flush_tlb_mask(d->domain_dirty_cpumask);
sh_set_toplevel_shadow(v, 0, gmfn, SH_type_l4_shadow);
#else
-#error This should never happen
+#error This should never happen
#endif
- ///
+ ///
/// v->arch.paging.shadow.l3table
///
#if SHADOW_PAGING_LEVELS == 3
@@ -4085,8 +4085,8 @@ sh_update_cr3(struct vcpu *v, int do_locking)
/* 3-on-3: make a PAE l3 that points at the four l2 pages */
smfn = pagetable_get_mfn(v->arch.shadow_table[i]);
#endif
- v->arch.paging.shadow.l3table[i] =
- (mfn_x(smfn) == 0)
+ v->arch.paging.shadow.l3table[i] =
+ (mfn_x(smfn) == 0)
? shadow_l3e_empty()
: shadow_l3e_from_mfn(smfn, _PAGE_PRESENT);
}
@@ -4165,7 +4165,7 @@ sh_update_cr3(struct vcpu *v, int do_locking)
/* Functions to revoke guest rights */
#if SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC
-int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
+int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
mfn_t smfn, unsigned long off)
{
int r;
@@ -4236,7 +4236,7 @@ static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
if ( !(shadow_l3e_get_flags(*sl3p) & _PAGE_PRESENT) )
return 0;
#else /* SHADOW_PAGING_LEVELS == 3 */
- sl3p = ((shadow_l3e_t *) v->arch.paging.shadow.l3table)
+ sl3p = ((shadow_l3e_t *) v->arch.paging.shadow.l3table)
+ shadow_l3_linear_offset(vaddr);
if ( !(shadow_l3e_get_flags(*sl3p) & _PAGE_PRESENT) )
return 0;
@@ -4272,20 +4272,20 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
shadow_l1e_t *sl1e;
int done = 0;
int flags;
-#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
+#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
mfn_t base_sl1mfn = sl1mfn; /* Because sl1mfn changes in the foreach */
#endif
-
- SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, done,
+
+ SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, done,
{
flags = shadow_l1e_get_flags(*sl1e);
- if ( (flags & _PAGE_PRESENT)
- && (flags & _PAGE_RW)
+ if ( (flags & _PAGE_PRESENT)
+ && (flags & _PAGE_RW)
&& (mfn_x(shadow_l1e_get_mfn(*sl1e)) == mfn_x(readonly_mfn)) )
{
shadow_l1e_t ro_sl1e = shadow_l1e_remove_flags(*sl1e, _PAGE_RW);
(void) shadow_set_l1e(v, sl1e, ro_sl1e, p2m_ram_rw, sl1mfn);
-#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
+#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
/* Remember the last shadow that we shot a writeable mapping in */
v->arch.paging.shadow.last_writeable_pte_smfn = mfn_x(base_sl1mfn);
#endif
@@ -4305,11 +4305,11 @@ int sh_rm_mappings_from_l1(struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn)
shadow_l1e_t *sl1e;
int done = 0;
int flags;
-
- SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, done,
+
+ SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, done,
{
flags = shadow_l1e_get_flags(*sl1e);
- if ( (flags & _PAGE_PRESENT)
+ if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l1e_get_mfn(*sl1e)) == mfn_x(target_mfn)) )
{
(void) shadow_set_l1e(v, sl1e, shadow_l1e_empty(),
@@ -4357,11 +4357,11 @@ int sh_remove_l1_shadow(struct vcpu *v, mfn_t sl2mfn, mfn_t sl1mfn)
shadow_l2e_t *sl2e;
int done = 0;
int flags;
-
- SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, done, v->domain,
+
+ SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, done, v->domain,
{
flags = shadow_l2e_get_flags(*sl2e);
- if ( (flags & _PAGE_PRESENT)
+ if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l2e_get_mfn(*sl2e)) == mfn_x(sl1mfn)) )
{
(void) shadow_set_l2e(v, sl2e, shadow_l2e_empty(), sl2mfn);
@@ -4380,11 +4380,11 @@ int sh_remove_l2_shadow(struct vcpu *v, mfn_t sl3mfn, mfn_t sl2mfn)
shadow_l3e_t *sl3e;
int done = 0;
int flags;
-
- SHADOW_FOREACH_L3E(sl3mfn, sl3e, 0, done,
+
+ SHADOW_FOREACH_L3E(sl3mfn, sl3e, 0, done,
{
flags = shadow_l3e_get_flags(*sl3e);
- if ( (flags & _PAGE_PRESENT)
+ if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l3e_get_mfn(*sl3e)) == mfn_x(sl2mfn)) )
{
(void) shadow_set_l3e(v, sl3e, shadow_l3e_empty(), sl3mfn);
@@ -4402,11 +4402,11 @@ int sh_remove_l3_shadow(struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn)
shadow_l4e_t *sl4e;
int done = 0;
int flags;
-
+
SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, done, v->domain,
{
flags = shadow_l4e_get_flags(*sl4e);
- if ( (flags & _PAGE_PRESENT)
+ if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l4e_get_mfn(*sl4e)) == mfn_x(sl3mfn)) )
{
(void) shadow_set_l4e(v, sl4e, shadow_l4e_empty(), sl4mfn);
@@ -4417,7 +4417,7 @@ int sh_remove_l3_shadow(struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn)
});
return done;
}
-#endif /* 64bit guest */
+#endif /* 64bit guest */
/**************************************************************************/
/* Function for the guest to inform us that a process is being torn
@@ -4517,7 +4517,7 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
#else
smfn = shadow_hash_lookup(v, mfn_x(gmfn), SH_type_l4_64_shadow);
#endif
-
+
if ( mfn_valid(smfn) )
{
mfn_to_page(gmfn)->shadow_flags |= SHF_pagetable_dying;
@@ -4557,7 +4557,7 @@ static mfn_t emulate_gva_to_mfn(struct vcpu *v,
/* Translate the VA to a GFN */
gfn = sh_gva_to_gfn(v, NULL, vaddr, &pfec);
- if ( gfn == INVALID_GFN )
+ if ( gfn == INVALID_GFN )
{
if ( is_hvm_vcpu(v) )
hvm_inject_page_fault(pfec, vaddr);
@@ -4596,7 +4596,7 @@ static mfn_t emulate_gva_to_mfn(struct vcpu *v,
return mfn;
}
-/* Check that the user is allowed to perform this write.
+/* Check that the user is allowed to perform this write.
* Returns a mapped pointer to write to, or NULL for error. */
#define MAPPING_UNHANDLEABLE ((void *)(unsigned long)X86EMUL_UNHANDLEABLE)
#define MAPPING_EXCEPTION ((void *)(unsigned long)X86EMUL_EXCEPTION)
@@ -4610,7 +4610,7 @@ static void *emulate_map_dest(struct vcpu *v,
void *map = NULL;
sh_ctxt->mfn1 = emulate_gva_to_mfn(v, vaddr, sh_ctxt);
- if ( !mfn_valid(sh_ctxt->mfn1) )
+ if ( !mfn_valid(sh_ctxt->mfn1) )
return ((mfn_x(sh_ctxt->mfn1) == BAD_GVA_TO_GFN) ?
MAPPING_EXCEPTION :
(mfn_x(sh_ctxt->mfn1) == READONLY_GFN) ?
@@ -4625,7 +4625,7 @@ static void *emulate_map_dest(struct vcpu *v,
return MAPPING_UNHANDLEABLE;
}
#endif
-
+
/* Unaligned writes mean probably this isn't a pagetable */
if ( vaddr & (bytes - 1) )
sh_remove_shadows(v, sh_ctxt->mfn1, 0, 0 /* Slow, can fail */ );
@@ -4636,11 +4636,11 @@ static void *emulate_map_dest(struct vcpu *v,
sh_ctxt->mfn2 = _mfn(INVALID_MFN);
map = sh_map_domain_page(sh_ctxt->mfn1) + (vaddr & ~PAGE_MASK);
}
- else
+ else
{
unsigned long mfns[2];
- /* Cross-page emulated writes are only supported for HVM guests;
+ /* Cross-page emulated writes are only supported for HVM guests;
* PV guests ought to know better */
if ( !is_hvm_vcpu(v) )
return MAPPING_UNHANDLEABLE;
@@ -4648,7 +4648,7 @@ static void *emulate_map_dest(struct vcpu *v,
/* This write crosses a page boundary. Translate the second page */
sh_ctxt->mfn2 = emulate_gva_to_mfn(v, (vaddr + bytes - 1) & PAGE_MASK,
sh_ctxt);
- if ( !mfn_valid(sh_ctxt->mfn2) )
+ if ( !mfn_valid(sh_ctxt->mfn2) )
return ((mfn_x(sh_ctxt->mfn2) == BAD_GVA_TO_GFN) ?
MAPPING_EXCEPTION :
(mfn_x(sh_ctxt->mfn2) == READONLY_GFN) ?
@@ -4656,7 +4656,7 @@ static void *emulate_map_dest(struct vcpu *v,
/* Cross-page writes mean probably not a pagetable */
sh_remove_shadows(v, sh_ctxt->mfn2, 0, 0 /* Slow, can fail */ );
-
+
mfns[0] = mfn_x(sh_ctxt->mfn1);
mfns[1] = mfn_x(sh_ctxt->mfn2);
map = vmap(mfns, 2);
@@ -4690,10 +4690,10 @@ static void emulate_unmap_dest(struct vcpu *v,
{
if ( ((unsigned long) addr & ((sizeof (guest_intpte_t)) - 1)) == 0 )
check_for_early_unshadow(v, sh_ctxt->mfn1);
- /* Don't reset the heuristic if we're writing zeros at non-aligned
+ /* Don't reset the heuristic if we're writing zeros at non-aligned
* addresses, otherwise it doesn't catch REP MOVSD on PAE guests */
}
- else
+ else
reset_early_unshadow(v);
/* We can avoid re-verifying the page contents after the write if:
@@ -4717,11 +4717,11 @@ static void emulate_unmap_dest(struct vcpu *v,
&& bytes <= 4)) )
{
/* Writes with this alignment constraint can't possibly cross pages */
- ASSERT(!mfn_valid(sh_ctxt->mfn2));
+ ASSERT(!mfn_valid(sh_ctxt->mfn2));
}
- else
+ else
#endif /* SHADOW_OPTIMIZATIONS & SHOPT_SKIP_VERIFY */
- {
+ {
if ( unlikely(mfn_valid(sh_ctxt->mfn2)) )
{
/* Validate as two writes, one to each page */
@@ -4742,7 +4742,7 @@ static void emulate_unmap_dest(struct vcpu *v,
paging_mark_dirty(v->domain, mfn_x(sh_ctxt->mfn2));
vunmap((void *)((unsigned long)addr & PAGE_MASK));
}
- else
+ else
sh_unmap_domain_page(addr);
atomic_inc(&v->domain->arch.paging.shadow.gtable_dirty_version);
@@ -4788,7 +4788,7 @@ sh_x86_emulate_write(struct vcpu *v, unsigned long vaddr, void *src,
}
static int
-sh_x86_emulate_cmpxchg(struct vcpu *v, unsigned long vaddr,
+sh_x86_emulate_cmpxchg(struct vcpu *v, unsigned long vaddr,
unsigned long old, unsigned long new,
unsigned int bytes, struct sh_emulate_ctxt *sh_ctxt)
{
@@ -4816,7 +4816,7 @@ sh_x86_emulate_cmpxchg(struct vcpu *v, unsigned long vaddr,
prev = ~old;
}
- if ( prev != old )
+ if ( prev != old )
rv = X86EMUL_CMPXCHG_FAILED;
SHADOW_DEBUG(EMULATE, "va %#lx was %#lx expected %#lx"
@@ -4865,28 +4865,28 @@ sh_x86_emulate_cmpxchg(struct vcpu *v, unsigned long vaddr,
} while (0)
static char * sh_audit_flags(struct vcpu *v, int level,
- int gflags, int sflags)
+ int gflags, int sflags)
/* Common code for auditing flag bits */
{
if ( (sflags & _PAGE_PRESENT) && !(gflags & _PAGE_PRESENT) )
return "shadow is present but guest is not present";
- if ( (sflags & _PAGE_GLOBAL) && !is_hvm_vcpu(v) )
+ if ( (sflags & _PAGE_GLOBAL) && !is_hvm_vcpu(v) )
return "global bit set in PV shadow";
if ( level == 2 && (sflags & _PAGE_PSE) )
return "PS bit set in shadow";
#if SHADOW_PAGING_LEVELS == 3
if ( level == 3 ) return NULL; /* All the other bits are blank in PAEl3 */
#endif
- if ( (sflags & _PAGE_PRESENT) && !(gflags & _PAGE_ACCESSED) )
+ if ( (sflags & _PAGE_PRESENT) && !(gflags & _PAGE_ACCESSED) )
return "accessed bit not propagated";
if ( (level == 1 || (level == 2 && (gflags & _PAGE_PSE)))
- && ((sflags & _PAGE_RW) && !(gflags & _PAGE_DIRTY)) )
+ && ((sflags & _PAGE_RW) && !(gflags & _PAGE_DIRTY)) )
return "dirty bit not propagated";
- if ( (sflags & _PAGE_USER) != (gflags & _PAGE_USER) )
+ if ( (sflags & _PAGE_USER) != (gflags & _PAGE_USER) )
return "user/supervisor bit does not match";
- if ( (sflags & _PAGE_NX_BIT) != (gflags & _PAGE_NX_BIT) )
+ if ( (sflags & _PAGE_NX_BIT) != (gflags & _PAGE_NX_BIT) )
return "NX bit does not match";
- if ( (sflags & _PAGE_RW) && !(gflags & _PAGE_RW) )
+ if ( (sflags & _PAGE_RW) && !(gflags & _PAGE_RW) )
return "shadow grants write access but guest does not";
return NULL;
}
@@ -4900,7 +4900,7 @@ int sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
p2m_type_t p2mt;
char *s;
int done = 0;
-
+
/* Follow the backpointer */
ASSERT(mfn_to_page(sl1mfn)->u.sh.head);
gl1mfn = backpointer(mfn_to_page(sl1mfn));
@@ -4917,32 +4917,32 @@ int sh_audit_l1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
gl1e = gp = sh_map_domain_page(gl1mfn);
SHADOW_FOREACH_L1E(sl1mfn, sl1e, &gl1e, done, {
- if ( sh_l1e_is_magic(*sl1e) )
+ if ( sh_l1e_is_magic(*sl1e) )
{
#if (SHADOW_OPTIMIZATIONS & SHOPT_FAST_FAULT_PATH)
if ( sh_l1e_is_gnp(*sl1e) )
{
if ( guest_l1e_get_flags(*gl1e) & _PAGE_PRESENT )
AUDIT_FAIL(1, "shadow is GNP magic but guest is present");
- }
- else
+ }
+ else
{
ASSERT(sh_l1e_is_mmio(*sl1e));
gfn = sh_l1e_mmio_get_gfn(*sl1e);
if ( gfn_x(gfn) != gfn_x(guest_l1e_get_gfn(*gl1e)) )
- AUDIT_FAIL(1, "shadow MMIO gfn is %" SH_PRI_gfn
+ AUDIT_FAIL(1, "shadow MMIO gfn is %" SH_PRI_gfn
" but guest gfn is %" SH_PRI_gfn,
gfn_x(gfn),
gfn_x(guest_l1e_get_gfn(*gl1e)));
}
#endif
}
- else
+ else
{
s = sh_audit_flags(v, 1, guest_l1e_get_flags(*gl1e),
shadow_l1e_get_flags(*sl1e));
if ( s ) AUDIT_FAIL(1, "%s", s);
-
+
if ( SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES_MFNS )
{
gfn = guest_l1e_get_gfn(*gl1e);
@@ -4972,12 +4972,12 @@ int sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, done, {
f = shadow_l1e_get_flags(*sl1e);
f &= ~(_PAGE_AVAIL0|_PAGE_AVAIL1|_PAGE_AVAIL2);
- if ( !(f == 0
+ if ( !(f == 0
|| f == (_PAGE_PRESENT|_PAGE_USER|_PAGE_RW|
- _PAGE_ACCESSED)
+ _PAGE_ACCESSED)
|| f == (_PAGE_PRESENT|_PAGE_USER|_PAGE_ACCESSED)
|| f == (_PAGE_PRESENT|_PAGE_USER|_PAGE_RW|
- _PAGE_ACCESSED|_PAGE_DIRTY)
+ _PAGE_ACCESSED|_PAGE_DIRTY)
|| f == (_PAGE_PRESENT|_PAGE_USER|_PAGE_ACCESSED|_PAGE_DIRTY)
|| sh_l1e_is_magic(*sl1e)) )
AUDIT_FAIL(1, "fl1e has bad flags");
@@ -5016,16 +5016,16 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
{
gfn = guest_l2e_get_gfn(*gl2e);
mfn = shadow_l2e_get_mfn(*sl2e);
- gmfn = (guest_l2e_get_flags(*gl2e) & _PAGE_PSE)
+ gmfn = (guest_l2e_get_flags(*gl2e) & _PAGE_PSE)
? get_fl1_shadow_status(v, gfn)
- : get_shadow_status(v,
- get_gfn_query_unlocked(v->domain, gfn_x(gfn),
+ : get_shadow_status(v,
+ get_gfn_query_unlocked(v->domain, gfn_x(gfn),
&p2mt), SH_type_l1_shadow);
if ( mfn_x(gmfn) != mfn_x(mfn) )
AUDIT_FAIL(2, "bad translation: gfn %" SH_PRI_gfn
" (--> %" PRI_mfn ")"
" --> %" PRI_mfn " != mfn %" PRI_mfn,
- gfn_x(gfn),
+ gfn_x(gfn),
(guest_l2e_get_flags(*gl2e) & _PAGE_PSE) ? 0
: mfn_x(get_gfn_query_unlocked(v->domain,
gfn_x(gfn), &p2mt)), mfn_x(gmfn), mfn_x(mfn));
@@ -5050,7 +5050,7 @@ int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
ASSERT(mfn_to_page(sl3mfn)->u.sh.head);
gl3mfn = backpointer(mfn_to_page(sl3mfn));
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Only L1's may be out of sync. */
if ( page_is_out_of_sync(mfn_to_page(gl3mfn)) )
AUDIT_FAIL_MIN(3, "gmfn %lx is out of sync", mfn_x(gl3mfn));
@@ -5099,7 +5099,7 @@ int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
ASSERT(mfn_to_page(sl4mfn)->u.sh.head);
gl4mfn = backpointer(mfn_to_page(sl4mfn));
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Only L1's may be out of sync. */
if ( page_is_out_of_sync(mfn_to_page(gl4mfn)) )
AUDIT_FAIL_MIN(4, "gmfn %lx is out of sync", mfn_x(gl4mfn));
@@ -5117,7 +5117,7 @@ int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
gfn = guest_l4e_get_gfn(*gl4e);
mfn = shadow_l4e_get_mfn(*sl4e);
gmfn = get_shadow_status(v, get_gfn_query_unlocked(
- v->domain, gfn_x(gfn), &p2mt),
+ v->domain, gfn_x(gfn), &p2mt),
SH_type_l3_shadow);
if ( mfn_x(gmfn) != mfn_x(mfn) )
AUDIT_FAIL(4, "bad translation: gfn %" SH_PRI_gfn
@@ -5139,7 +5139,7 @@ int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
/* Entry points into this mode of the shadow code.
* This will all be mangled by the preprocessor to uniquify everything. */
const struct paging_mode sh_paging_mode = {
- .page_fault = sh_page_fault,
+ .page_fault = sh_page_fault,
.invlpg = sh_invlpg,
.gva_to_gfn = sh_gva_to_gfn,
.update_cr3 = sh_update_cr3,
@@ -5168,5 +5168,5 @@ const struct paging_mode sh_paging_mode = {
* c-file-style: "BSD"
* c-basic-offset: 4
* indent-tabs-mode: nil
- * End:
+ * End:
*/
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 835121e..7f829fd 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -21,42 +21,42 @@
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
-extern int
+extern int
SHADOW_INTERNAL_NAME(sh_map_and_validate_gl1e, GUEST_LEVELS)(
struct vcpu *v, mfn_t gl1mfn, void *new_gl1p, u32 size);
-extern int
+extern int
SHADOW_INTERNAL_NAME(sh_map_and_validate_gl2e, GUEST_LEVELS)(
struct vcpu *v, mfn_t gl2mfn, void *new_gl2p, u32 size);
-extern int
+extern int
SHADOW_INTERNAL_NAME(sh_map_and_validate_gl2he, GUEST_LEVELS)(
struct vcpu *v, mfn_t gl2mfn, void *new_gl2p, u32 size);
-extern int
+extern int
SHADOW_INTERNAL_NAME(sh_map_and_validate_gl3e, GUEST_LEVELS)(
struct vcpu *v, mfn_t gl3mfn, void *new_gl3p, u32 size);
-extern int
+extern int
SHADOW_INTERNAL_NAME(sh_map_and_validate_gl4e, GUEST_LEVELS)(
struct vcpu *v, mfn_t gl4mfn, void *new_gl4p, u32 size);
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, GUEST_LEVELS)(
struct vcpu *v, mfn_t smfn);
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, GUEST_LEVELS)(
struct vcpu *v, mfn_t smfn);
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_destroy_l3_shadow, GUEST_LEVELS)(
struct vcpu *v, mfn_t smfn);
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_destroy_l4_shadow, GUEST_LEVELS)(
struct vcpu *v, mfn_t smfn);
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_unhook_32b_mappings, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl2mfn, int user_only);
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_unhook_pae_mappings, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl3mfn, int user_only);
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_unhook_64b_mappings, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl4mfn, int user_only);
@@ -82,19 +82,19 @@ SHADOW_INTERNAL_NAME(sh_remove_l3_shadow, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn);
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES
-int
+int
SHADOW_INTERNAL_NAME(sh_audit_l1_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl1mfn, mfn_t x);
-int
+int
SHADOW_INTERNAL_NAME(sh_audit_fl1_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl1mfn, mfn_t x);
-int
+int
SHADOW_INTERNAL_NAME(sh_audit_l2_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl2mfn, mfn_t x);
-int
+int
SHADOW_INTERNAL_NAME(sh_audit_l3_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl3mfn, mfn_t x);
-int
+int
SHADOW_INTERNAL_NAME(sh_audit_l4_table, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl4mfn, mfn_t x);
#endif
@@ -117,7 +117,7 @@ extern const struct paging_mode
SHADOW_INTERNAL_NAME(sh_paging_mode, GUEST_LEVELS);
#if SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC
-extern void
+extern void
SHADOW_INTERNAL_NAME(sh_resync_l1, GUEST_LEVELS)
(struct vcpu *v, mfn_t gmfn, mfn_t snpmfn);
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index 72518fd..4b69626 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -139,7 +139,7 @@ enum {
#endif
/******************************************************************************
- * Auditing routines
+ * Auditing routines
*/
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES_FULL
@@ -191,27 +191,27 @@ extern void shadow_audit_tables(struct vcpu *v);
#define SH_type_oos_snapshot (16U) /* in use as OOS snapshot */
#define SH_type_unused (17U)
-/*
+/*
* What counts as a pinnable shadow?
*/
-static inline int sh_type_is_pinnable(struct vcpu *v, unsigned int t)
+static inline int sh_type_is_pinnable(struct vcpu *v, unsigned int t)
{
- /* Top-level shadow types in each mode can be pinned, so that they
+ /* Top-level shadow types in each mode can be pinned, so that they
* persist even when not currently in use in a guest CR3 */
if ( t == SH_type_l2_32_shadow
|| t == SH_type_l2_pae_shadow
- || t == SH_type_l2h_pae_shadow
+ || t == SH_type_l2h_pae_shadow
|| t == SH_type_l4_64_shadow )
return 1;
-#if (SHADOW_OPTIMIZATIONS & SHOPT_LINUX_L3_TOPLEVEL)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_LINUX_L3_TOPLEVEL)
/* Early 64-bit linux used three levels of pagetables for the guest
* and context switched by changing one l4 entry in a per-cpu l4
* page. When we're shadowing those kernels, we have to pin l3
* shadows so they don't just evaporate on every context switch.
- * For all other guests, we'd rather use the up-pointer field in l3s. */
- if ( unlikely((v->domain->arch.paging.shadow.opt_flags & SHOPT_LINUX_L3_TOPLEVEL)
+ * For all other guests, we'd rather use the up-pointer field in l3s. */
+ if ( unlikely((v->domain->arch.paging.shadow.opt_flags & SHOPT_LINUX_L3_TOPLEVEL)
&& t == SH_type_l3_64_shadow) )
return 1;
#endif
@@ -220,7 +220,7 @@ static inline int sh_type_is_pinnable(struct vcpu *v, unsigned int t)
return 0;
}
-static inline int sh_type_has_up_pointer(struct vcpu *v, unsigned int t)
+static inline int sh_type_has_up_pointer(struct vcpu *v, unsigned int t)
{
/* Multi-page shadows don't have up-pointers */
if ( t == SH_type_l1_32_shadow
@@ -271,9 +271,9 @@ static inline void sh_terminate_list(struct page_list_head *tmp_list)
#define SHF_L1_ANY (SHF_L1_32|SHF_L1_PAE|SHF_L1_64)
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Marks a guest L1 page table which is shadowed but not write-protected.
- * If set, then *only* L1 shadows (SHF_L1_*) are allowed.
+ * If set, then *only* L1 shadows (SHF_L1_*) are allowed.
*
* out_of_sync indicates that the shadow tables may not reflect the
* guest tables. If it is clear, then the shadow tables *must* reflect
@@ -281,9 +281,9 @@ static inline void sh_terminate_list(struct page_list_head *tmp_list)
*
* oos_may_write indicates that a page may have writable mappings.
*
- * Most of the time the flags are synonymous. There is a short period of time
- * during resync that oos_may_write is clear but out_of_sync is not. If a
- * codepath is called during that time and is sensitive to oos issues, it may
+ * Most of the time the flags are synonymous. There is a short period of time
+ * during resync that oos_may_write is clear but out_of_sync is not. If a
+ * codepath is called during that time and is sensitive to oos issues, it may
* need to use the second flag.
*/
#define SHF_out_of_sync (1u<<30)
@@ -303,27 +303,27 @@ static inline int sh_page_has_multiple_shadows(struct page_info *pg)
return ( (shadows & ~(1UL << find_first_set_bit(shadows))) != 0 );
}
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* The caller must verify this is reasonable to call; i.e., valid mfn,
* domain is translated, &c */
-static inline int page_is_out_of_sync(struct page_info *p)
+static inline int page_is_out_of_sync(struct page_info *p)
{
return (p->count_info & PGC_page_table)
&& (p->shadow_flags & SHF_out_of_sync);
}
-static inline int mfn_is_out_of_sync(mfn_t gmfn)
+static inline int mfn_is_out_of_sync(mfn_t gmfn)
{
return page_is_out_of_sync(mfn_to_page(mfn_x(gmfn)));
}
-static inline int page_oos_may_write(struct page_info *p)
+static inline int page_oos_may_write(struct page_info *p)
{
return (p->count_info & PGC_page_table)
&& (p->shadow_flags & SHF_oos_may_write);
}
-static inline int mfn_oos_may_write(mfn_t gmfn)
+static inline int mfn_oos_may_write(mfn_t gmfn)
{
return page_oos_may_write(mfn_to_page(mfn_x(gmfn)));
}
@@ -339,14 +339,14 @@ shadow_size(unsigned int shadow_type)
}
/******************************************************************************
- * Various function declarations
+ * Various function declarations
*/
/* Hash table functions */
mfn_t shadow_hash_lookup(struct vcpu *v, unsigned long n, unsigned int t);
-void shadow_hash_insert(struct vcpu *v,
+void shadow_hash_insert(struct vcpu *v,
unsigned long n, unsigned int t, mfn_t smfn);
-void shadow_hash_delete(struct vcpu *v,
+void shadow_hash_delete(struct vcpu *v,
unsigned long n, unsigned int t, mfn_t smfn);
/* shadow promotion */
@@ -355,7 +355,7 @@ void shadow_demote(struct vcpu *v, mfn_t gmfn, u32 type);
/* Shadow page allocation functions */
void shadow_prealloc(struct domain *d, u32 shadow_type, unsigned int count);
-mfn_t shadow_alloc(struct domain *d,
+mfn_t shadow_alloc(struct domain *d,
u32 shadow_type,
unsigned long backpointer);
void shadow_free(struct domain *d, mfn_t smfn);
@@ -367,11 +367,11 @@ void sh_install_xen_entries_in_l4(struct vcpu *v, mfn_t gl4mfn, mfn_t sl4mfn);
int sh_validate_guest_entry(struct vcpu *v, mfn_t gmfn, void *entry, u32 size);
/* Update the shadows in response to a pagetable write from a HVM guest */
-void sh_validate_guest_pt_write(struct vcpu *v, mfn_t gmfn,
+void sh_validate_guest_pt_write(struct vcpu *v, mfn_t gmfn,
void *entry, u32 size);
/* Remove all writeable mappings of a guest frame from the shadows.
- * Returns non-zero if we need to flush TLBs.
+ * Returns non-zero if we need to flush TLBs.
* level and fault_addr desribe how we found this to be a pagetable;
* level==0 means we have some other reason for revoking write access. */
extern int sh_remove_write_access(struct vcpu *v, mfn_t readonly_mfn,
@@ -437,7 +437,7 @@ mfn_t oos_snapshot_lookup(struct vcpu *v, mfn_t gmfn);
#endif /* (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC) */
-/* Reset the up-pointers of every L3 shadow to 0.
+/* Reset the up-pointers of every L3 shadow to 0.
* This is called when l3 shadows stop being pinnable, to clear out all
* the list-head bits so the up-pointer field is properly inititalised. */
void sh_reset_l3_up_pointers(struct vcpu *v);
@@ -455,7 +455,7 @@ void sh_reset_l3_up_pointers(struct vcpu *v);
/******************************************************************************
- * MFN/page-info handling
+ * MFN/page-info handling
*/
/* Override macros from asm/page.h to make them work with mfn_t */
@@ -496,9 +496,9 @@ sh_mfn_is_a_page_table(mfn_t gmfn)
return 0;
owner = page_get_owner(page);
- if ( owner && shadow_mode_refcounts(owner)
+ if ( owner && shadow_mode_refcounts(owner)
&& (page->count_info & PGC_page_table) )
- return 1;
+ return 1;
type_info = page->u.inuse.type_info & PGT_type_mask;
return type_info && (type_info <= PGT_l4_page_table);
@@ -511,8 +511,8 @@ sh_map_domain_page(mfn_t mfn)
return map_domain_page(mfn_x(mfn));
}
-static inline void
-sh_unmap_domain_page(void *p)
+static inline void
+sh_unmap_domain_page(void *p)
{
unmap_domain_page(p);
}
@@ -523,8 +523,8 @@ sh_map_domain_page_global(mfn_t mfn)
return map_domain_page_global(mfn_x(mfn));
}
-static inline void
-sh_unmap_domain_page_global(void *p)
+static inline void
+sh_unmap_domain_page_global(void *p)
{
unmap_domain_page_global(p);
}
@@ -534,9 +534,9 @@ sh_unmap_domain_page_global(void *p)
void sh_destroy_shadow(struct vcpu *v, mfn_t smfn);
-/* Increase the refcount of a shadow page. Arguments are the mfn to refcount,
+/* Increase the refcount of a shadow page. Arguments are the mfn to refcount,
* and the physical address of the shadow entry that holds the ref (or zero
- * if the ref is held by something else).
+ * if the ref is held by something else).
* Returns 0 for failure, 1 for success. */
static inline int sh_get_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
{
@@ -555,16 +555,16 @@ static inline int sh_get_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
__backpointer(sp), mfn_x(smfn));
return 0;
}
-
+
/* Guarded by the paging lock, so no need for atomic update */
sp->u.sh.count = nx;
/* We remember the first shadow entry that points to each shadow. */
- if ( entry_pa != 0
+ if ( entry_pa != 0
&& sh_type_has_up_pointer(v, sp->u.sh.type)
- && sp->up == 0 )
+ && sp->up == 0 )
sp->up = entry_pa;
-
+
return 1;
}
@@ -581,15 +581,15 @@ static inline void sh_put_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
ASSERT(!(sp->count_info & PGC_count_mask));
/* If this is the entry in the up-pointer, remove it */
- if ( entry_pa != 0
+ if ( entry_pa != 0
&& sh_type_has_up_pointer(v, sp->u.sh.type)
- && sp->up == entry_pa )
+ && sp->up == entry_pa )
sp->up = 0;
x = sp->u.sh.count;
nx = x - 1;
- if ( unlikely(x == 0) )
+ if ( unlikely(x == 0) )
{
SHADOW_ERROR("shadow ref underflow, smfn=%lx oc=%08x t=%#x\n",
mfn_x(smfn), sp->u.sh.count, sp->u.sh.type);
@@ -599,12 +599,12 @@ static inline void sh_put_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
/* Guarded by the paging lock, so no need for atomic update */
sp->u.sh.count = nx;
- if ( unlikely(nx == 0) )
+ if ( unlikely(nx == 0) )
sh_destroy_shadow(v, smfn);
}
-/* Walk the list of pinned shadows, from the tail forwards,
+/* Walk the list of pinned shadows, from the tail forwards,
* skipping the non-head-page entries */
static inline struct page_info *
prev_pinned_shadow(const struct page_info *page,
@@ -747,7 +747,7 @@ struct sh_emulate_ctxt {
mfn_t mfn1, mfn2;
#if (SHADOW_OPTIMIZATIONS & SHOPT_SKIP_VERIFY)
- /* Special case for avoiding having to verify writes: remember
+ /* Special case for avoiding having to verify writes: remember
* whether the old value had its low bit (_PAGE_PRESENT) clear. */
int low_bit_was_clear:1;
#endif
@@ -762,12 +762,12 @@ struct segment_register *hvm_get_seg_reg(
#if (SHADOW_OPTIMIZATIONS & SHOPT_VIRTUAL_TLB)
/**************************************************************************/
-/* Virtual TLB entries
+/* Virtual TLB entries
*
- * We keep a cache of virtual-to-physical translations that we have seen
- * since the last TLB flush. This is safe to use for frame translations,
+ * We keep a cache of virtual-to-physical translations that we have seen
+ * since the last TLB flush. This is safe to use for frame translations,
* but callers need to re-check the actual guest tables if the lookup fails.
- *
+ *
* Lookups and updates are protected by a per-vTLB (and hence per-vcpu)
* lock. This lock is held *only* while reading or writing the table,
* so it is safe to take in any non-interrupt context. Most lookups
@@ -785,7 +785,7 @@ struct shadow_vtlb {
};
/* Call whenever the guest flushes hit actual TLB */
-static inline void vtlb_flush(struct vcpu *v)
+static inline void vtlb_flush(struct vcpu *v)
{
spin_lock(&v->arch.paging.vtlb_lock);
memset(v->arch.paging.vtlb, 0, VTLB_ENTRIES * sizeof (struct shadow_vtlb));
@@ -801,7 +801,7 @@ static inline int vtlb_hash(unsigned long page_number)
static inline void vtlb_insert(struct vcpu *v, unsigned long page,
unsigned long frame, uint32_t pfec)
{
- struct shadow_vtlb entry =
+ struct shadow_vtlb entry =
{ .page_number = page, .frame_number = frame, .pfec = pfec };
spin_lock(&v->arch.paging.vtlb_lock);
v->arch.paging.vtlb[vtlb_hash(page)] = entry;
@@ -818,7 +818,7 @@ static inline unsigned long vtlb_lookup(struct vcpu *v,
spin_lock(&v->arch.paging.vtlb_lock);
if ( v->arch.paging.vtlb[i].pfec != 0
- && v->arch.paging.vtlb[i].page_number == page_number
+ && v->arch.paging.vtlb[i].page_number == page_number
/* Any successful walk that had at least these pfec bits is OK */
&& (v->arch.paging.vtlb[i].pfec & pfec) == pfec )
{
@@ -833,7 +833,7 @@ static inline int sh_check_page_has_no_refs(struct page_info *page)
{
unsigned long count = read_atomic(&page->count_info);
return ( (count & PGC_count_mask) ==
- ((count & PGC_allocated) ? 1 : 0) );
+ ((count & PGC_allocated) ? 1 : 0) );
}
#endif /* _XEN_SHADOW_PRIVATE_H */
diff --git a/xen/arch/x86/mm/shadow/types.h b/xen/arch/x86/mm/shadow/types.h
index 953d168..7391b3c 100644
--- a/xen/arch/x86/mm/shadow/types.h
+++ b/xen/arch/x86/mm/shadow/types.h
@@ -1,20 +1,20 @@
/******************************************************************************
* arch/x86/mm/shadow/types.h
- *
+ *
* Parts of this code are Copyright (c) 2006 by XenSource Inc.
* Parts of this code are Copyright (c) 2006 by Michael A Fetterman
* Parts based on earlier work by Michael A Fetterman, Ian Pratt et al.
- *
+ *
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
- *
+ *
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
+ *
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
@@ -31,7 +31,7 @@
#define SHADOW_PAGING_LEVELS 3
#endif
-/*
+/*
* Define various types for handling pagetabels, based on these options:
* SHADOW_PAGING_LEVELS : Number of levels of shadow pagetables
* GUEST_PAGING_LEVELS : Number of levels of guest pagetables
@@ -101,14 +101,14 @@ static inline shadow_l1e_t
shadow_l1e_remove_flags(shadow_l1e_t sl1e, u32 flags)
{ l1e_remove_flags(sl1e, flags); return sl1e; }
-static inline shadow_l1e_t shadow_l1e_empty(void)
+static inline shadow_l1e_t shadow_l1e_empty(void)
{ return l1e_empty(); }
-static inline shadow_l2e_t shadow_l2e_empty(void)
+static inline shadow_l2e_t shadow_l2e_empty(void)
{ return l2e_empty(); }
-static inline shadow_l3e_t shadow_l3e_empty(void)
+static inline shadow_l3e_t shadow_l3e_empty(void)
{ return l3e_empty(); }
#if SHADOW_PAGING_LEVELS >= 4
-static inline shadow_l4e_t shadow_l4e_empty(void)
+static inline shadow_l4e_t shadow_l4e_empty(void)
{ return l4e_empty(); }
#endif
@@ -141,10 +141,10 @@ static inline shadow_l4e_t shadow_l4e_from_mfn(mfn_t mfn, u32 flags)
#define shadow_l4_linear_offset(_a) \
(((_a) & VADDR_MASK) >> SHADOW_L4_PAGETABLE_SHIFT)
-/* Where to find each level of the linear mapping. For PV guests, we use
- * the shadow linear-map self-entry as many times as we need. For HVM
- * guests, the shadow doesn't have a linear-map self-entry so we must use
- * the monitor-table's linear-map entry N-1 times and then the shadow-map
+/* Where to find each level of the linear mapping. For PV guests, we use
+ * the shadow linear-map self-entry as many times as we need. For HVM
+ * guests, the shadow doesn't have a linear-map self-entry so we must use
+ * the monitor-table's linear-map entry N-1 times and then the shadow-map
* entry once. */
#define __sh_linear_l1_table ((shadow_l1e_t *)(SH_LINEAR_PT_VIRT_START))
#define __sh_linear_l2_table ((shadow_l2e_t *) \
@@ -304,12 +304,12 @@ static inline int sh_l1e_is_magic(shadow_l1e_t sl1e)
}
/* Guest not present: a single magic value */
-static inline shadow_l1e_t sh_l1e_gnp(void)
+static inline shadow_l1e_t sh_l1e_gnp(void)
{
return (shadow_l1e_t){ -1ULL };
}
-static inline int sh_l1e_is_gnp(shadow_l1e_t sl1e)
+static inline int sh_l1e_is_gnp(shadow_l1e_t sl1e)
{
return (sl1e.l1 == sh_l1e_gnp().l1);
}
@@ -323,24 +323,24 @@ static inline int sh_l1e_is_gnp(shadow_l1e_t sl1e)
#define SH_L1E_MMIO_GFN_MASK 0x00000000fffffff0ULL
#define SH_L1E_MMIO_GFN_SHIFT 4
-static inline shadow_l1e_t sh_l1e_mmio(gfn_t gfn, u32 gflags)
+static inline shadow_l1e_t sh_l1e_mmio(gfn_t gfn, u32 gflags)
{
- return (shadow_l1e_t) { (SH_L1E_MMIO_MAGIC
- | (gfn_x(gfn) << SH_L1E_MMIO_GFN_SHIFT)
+ return (shadow_l1e_t) { (SH_L1E_MMIO_MAGIC
+ | (gfn_x(gfn) << SH_L1E_MMIO_GFN_SHIFT)
| (gflags & (_PAGE_USER|_PAGE_RW))) };
}
-static inline int sh_l1e_is_mmio(shadow_l1e_t sl1e)
+static inline int sh_l1e_is_mmio(shadow_l1e_t sl1e)
{
return ((sl1e.l1 & SH_L1E_MMIO_MAGIC_MASK) == SH_L1E_MMIO_MAGIC);
}
-static inline gfn_t sh_l1e_mmio_get_gfn(shadow_l1e_t sl1e)
+static inline gfn_t sh_l1e_mmio_get_gfn(shadow_l1e_t sl1e)
{
return _gfn((sl1e.l1 & SH_L1E_MMIO_GFN_MASK) >> SH_L1E_MMIO_GFN_SHIFT);
}
-static inline u32 sh_l1e_mmio_get_flags(shadow_l1e_t sl1e)
+static inline u32 sh_l1e_mmio_get_flags(shadow_l1e_t sl1e)
{
return (u32)((sl1e.l1 & (_PAGE_USER|_PAGE_RW)));
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 02/20] x86/shadow: Rename hash_foreach() to hash_vcpu_foreach()
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
2015-02-12 17:16 ` [PATCH 01/20] x86/shadow: Whitespace cleanup Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 03/20] x86/shadow: Introduce 'd' pointers and clean up use of 'v->domain' Andrew Cooper
` (19 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
A later change requires the introduction of a domain variant.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 31 +++++++++++++++----------------
1 file changed, 15 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 502e0d8..2e4954f 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1999,12 +1999,11 @@ void shadow_hash_delete(struct vcpu *v, unsigned long n, unsigned int t,
sh_hash_audit_bucket(d, key);
}
-typedef int (*hash_callback_t)(struct vcpu *v, mfn_t smfn, mfn_t other_mfn);
+typedef int (*hash_vcpu_callback_t)(struct vcpu *v, mfn_t smfn, mfn_t other_mfn);
-static void hash_foreach(struct vcpu *v,
- unsigned int callback_mask,
- const hash_callback_t callbacks[],
- mfn_t callback_mfn)
+static void hash_vcpu_foreach(struct vcpu *v, unsigned int callback_mask,
+ const hash_vcpu_callback_t callbacks[],
+ mfn_t callback_mfn)
/* Walk the hash table looking at the types of the entries and
* calling the appropriate callback function for each entry.
* The mask determines which shadow types we call back for, and the array
@@ -2140,7 +2139,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
unsigned long fault_addr)
{
/* Dispatch table for getting per-type functions */
- static const hash_callback_t callbacks[SH_type_unused] = {
+ static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, 2), /* l1_32 */
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, 2), /* fl1_32 */
@@ -2334,7 +2333,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
perfc_incr(shadow_writeable_bf_1);
else
perfc_incr(shadow_writeable_bf);
- hash_foreach(v, callback_mask, callbacks, gmfn);
+ hash_vcpu_foreach(v, callback_mask, callbacks, gmfn);
/* If that didn't catch the mapping, then there's some non-pagetable
* mapping -- ioreq page, grant mapping, &c. */
@@ -2390,7 +2389,7 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
struct page_info *page = mfn_to_page(gmfn);
/* Dispatch table for getting per-type functions */
- static const hash_callback_t callbacks[SH_type_unused] = {
+ static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, 2), /* l1_32 */
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, 2), /* fl1_32 */
@@ -2432,7 +2431,7 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
/* Brute-force search of all the shadows, by walking the hash */
perfc_incr(shadow_mappings_bf);
- hash_foreach(v, callback_mask, callbacks, gmfn);
+ hash_vcpu_foreach(v, callback_mask, callbacks, gmfn);
/* If that didn't catch the mapping, something is very wrong */
if ( !sh_check_page_has_no_refs(page) )
@@ -2533,7 +2532,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
/* Dispatch table for getting per-type functions: each level must
* be called with the function to remove a lower-level shadow. */
- static const hash_callback_t callbacks[SH_type_unused] = {
+ static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
NULL, /* l1_32 */
NULL, /* fl1_32 */
@@ -2594,7 +2593,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
perfc_incr(shadow_unshadow);
/* Lower-level shadows need to be excised from upper-level shadows.
- * This call to hash_foreach() looks dangerous but is in fact OK: each
+ * This call to hash_vcpu_foreach() looks dangerous but is in fact OK: each
* call will remove at most one shadow, and terminate immediately when
* it does remove it, so we never walk the hash after doing a deletion. */
#define DO_UNSHADOW(_type) do { \
@@ -2617,7 +2616,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
if( !fast \
&& (pg->count_info & PGC_page_table) \
&& (pg->shadow_flags & (1 << t)) ) \
- hash_foreach(v, masks[t], callbacks, smfn); \
+ hash_vcpu_foreach(v, masks[t], callbacks, smfn); \
} while (0)
DO_UNSHADOW(SH_type_l2_32_shadow);
@@ -2678,7 +2677,7 @@ static int sh_clear_up_pointer(struct vcpu *v, mfn_t smfn, mfn_t unused)
void sh_reset_l3_up_pointers(struct vcpu *v)
{
- static const hash_callback_t callbacks[SH_type_unused] = {
+ static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
NULL, /* l1_32 */
NULL, /* fl1_32 */
@@ -2698,7 +2697,7 @@ void sh_reset_l3_up_pointers(struct vcpu *v)
};
static const unsigned int callback_mask = SHF_L3_64;
- hash_foreach(v, callback_mask, callbacks, _mfn(INVALID_MFN));
+ hash_vcpu_foreach(v, callback_mask, callbacks, _mfn(INVALID_MFN));
}
@@ -3729,7 +3728,7 @@ int shadow_domctl(struct domain *d,
void shadow_audit_tables(struct vcpu *v)
{
/* Dispatch table for getting per-type functions */
- static const hash_callback_t callbacks[SH_type_unused] = {
+ static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
SHADOW_INTERNAL_NAME(sh_audit_l1_table, 2), /* l1_32 */
SHADOW_INTERNAL_NAME(sh_audit_fl1_table, 2), /* fl1_32 */
@@ -3771,7 +3770,7 @@ void shadow_audit_tables(struct vcpu *v)
}
}
- hash_foreach(v, mask, callbacks, _mfn(INVALID_MFN));
+ hash_vcpu_foreach(v, mask, callbacks, _mfn(INVALID_MFN));
}
#endif /* Shadow audit */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 03/20] x86/shadow: Introduce 'd' pointers and clean up use of 'v->domain'
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
2015-02-12 17:16 ` [PATCH 01/20] x86/shadow: Whitespace cleanup Andrew Cooper
2015-02-12 17:16 ` [PATCH 02/20] x86/shadow: Rename hash_foreach() to hash_vcpu_foreach() Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 04/20] x86/shadow: Only apply shadow heuristics when in guest context Andrew Cooper
` (18 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
All of the introduced domain pointers will eventually be removed, but doing
this mechanical cleanup here allows the subsequent patches which change
function prototypes to be smaller and more clear.
In addition, swap some use of is_pv_32on64_vcpu(v) for is_pv_32on64_domain(d).
No functional change.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 49 +++++++------
xen/arch/x86/mm/shadow/multi.c | 146 ++++++++++++++++++++++----------------
xen/arch/x86/mm/shadow/private.h | 6 +-
3 files changed, 116 insertions(+), 85 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 2e4954f..3b5ef19 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -665,6 +665,7 @@ void oos_fixup_add(struct vcpu *v, mfn_t gmfn,
static int oos_remove_write_access(struct vcpu *v, mfn_t gmfn,
struct oos_fixup *fixup)
{
+ struct domain *d = v->domain;
int ftlb = 0;
ftlb |= oos_fixup_flush_gmfn(v, gmfn, fixup);
@@ -690,7 +691,7 @@ static int oos_remove_write_access(struct vcpu *v, mfn_t gmfn,
}
if ( ftlb )
- flush_tlb_mask(v->domain->domain_dirty_cpumask);
+ flush_tlb_mask(d->domain_dirty_cpumask);
return 0;
}
@@ -991,6 +992,7 @@ int sh_unsync(struct vcpu *v, mfn_t gmfn)
*/
void shadow_promote(struct vcpu *v, mfn_t gmfn, unsigned int type)
{
+ struct domain *d = v->domain;
struct page_info *page = mfn_to_page(gmfn);
ASSERT(mfn_valid(gmfn));
@@ -1004,7 +1006,7 @@ void shadow_promote(struct vcpu *v, mfn_t gmfn, unsigned int type)
/* We should never try to promote a gmfn that has writeable mappings */
ASSERT((page->u.inuse.type_info & PGT_type_mask) != PGT_writable_page
|| (page->u.inuse.type_info & PGT_count_mask) == 0
- || v->domain->is_shutting_down);
+ || d->is_shutting_down);
/* Is the page already shadowed? */
if ( !test_and_set_bit(_PGC_page_table, &page->count_info) )
@@ -2056,6 +2058,7 @@ static void hash_vcpu_foreach(struct vcpu *v, unsigned int callback_mask,
void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
{
+ struct domain *d = v->domain;
struct page_info *sp = mfn_to_page(smfn);
unsigned int t = sp->u.sh.type;
@@ -2068,9 +2071,8 @@ void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
t == SH_type_fl1_pae_shadow ||
t == SH_type_fl1_64_shadow ||
t == SH_type_monitor_table ||
- (is_pv_32on64_vcpu(v) && t == SH_type_l4_64_shadow) ||
- (page_get_owner(mfn_to_page(backpointer(sp)))
- == v->domain));
+ (is_pv_32on64_domain(d) && t == SH_type_l4_64_shadow) ||
+ (page_get_owner(mfn_to_page(backpointer(sp))) == d));
/* The down-shifts here are so that the switch statement is on nice
* small numbers that the compiler will enjoy */
@@ -2098,7 +2100,7 @@ void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, 4)(v, smfn);
break;
case SH_type_l2h_64_shadow:
- ASSERT(is_pv_32on64_vcpu(v));
+ ASSERT(is_pv_32on64_domain(d));
/* Fall through... */
case SH_type_l2_64_shadow:
SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, 4)(v, smfn);
@@ -2166,15 +2168,16 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
| SHF_L1_64
| SHF_FL1_64
;
+ struct domain *d = v->domain;
struct page_info *pg = mfn_to_page(gmfn);
- ASSERT(paging_locked_by_me(v->domain));
+ ASSERT(paging_locked_by_me(d));
/* Only remove writable mappings if we are doing shadow refcounts.
* In guest refcounting, we trust Xen to already be restricting
* all the writes to the guest page tables, so we do not need to
* do more. */
- if ( !shadow_mode_refcounts(v->domain) )
+ if ( !shadow_mode_refcounts(d) )
return 0;
/* Early exit if it's already a pagetable, or otherwise not writeable */
@@ -2198,7 +2201,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
SHADOW_ERROR("can't remove write access to mfn %lx, type_info is %"
PRtype_info "\n",
mfn_x(gmfn), mfn_to_page(gmfn)->u.inuse.type_info);
- domain_crash(v->domain);
+ domain_crash(d);
}
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
@@ -2226,7 +2229,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
GUESS(0xC0000000UL + (fault_addr >> 10), 1);
/* Linux lowmem: first 896MB is mapped 1-to-1 above 0xC0000000 */
- if ((gfn = mfn_to_gfn(v->domain, gmfn)) < 0x38000 )
+ if ((gfn = mfn_to_gfn(d, gmfn)) < 0x38000 )
GUESS(0xC0000000UL + (gfn << PAGE_SHIFT), 4);
/* FreeBSD: Linear map at 0xBFC00000 */
@@ -2244,7 +2247,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
}
/* Linux lowmem: first 896MB is mapped 1-to-1 above 0xC0000000 */
- if ((gfn = mfn_to_gfn(v->domain, gmfn)) < 0x38000 )
+ if ((gfn = mfn_to_gfn(d, gmfn)) < 0x38000 )
GUESS(0xC0000000UL + (gfn << PAGE_SHIFT), 4);
/* FreeBSD PAE: Linear map at 0xBF800000 */
@@ -2272,7 +2275,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
/* 64bit Linux direct map at 0xffff880000000000; older kernels
* had it at 0xffff810000000000, and older kernels yet had it
* at 0x0000010000000000UL */
- gfn = mfn_to_gfn(v->domain, gmfn);
+ gfn = mfn_to_gfn(d, gmfn);
GUESS(0xffff880000000000UL + (gfn << PAGE_SHIFT), 4);
GUESS(0xffff810000000000UL + (gfn << PAGE_SHIFT), 4);
GUESS(0x0000010000000000UL + (gfn << PAGE_SHIFT), 4);
@@ -2345,7 +2348,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
SHADOW_ERROR("can't remove write access to mfn %lx: guest has "
"%lu special-use mappings of it\n", mfn_x(gmfn),
(mfn_to_page(gmfn)->u.inuse.type_info&PGT_count_mask));
- domain_crash(v->domain);
+ domain_crash(d);
}
/* We killed at least one writeable mapping, so must flush TLBs. */
@@ -2386,6 +2389,7 @@ int sh_remove_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
{
+ struct domain *d = v->domain;
struct page_info *page = mfn_to_page(gmfn);
/* Dispatch table for getting per-type functions */
@@ -2424,7 +2428,7 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
/* Although this is an externally visible function, we do not know
* whether the paging lock will be held when it is called (since it
* can be called via put_page_type when we clear a shadow l1e).*/
- paging_lock_recursive(v->domain);
+ paging_lock_recursive(d);
/* XXX TODO:
* Heuristics for finding the (probably) single mapping of this gmfn */
@@ -2441,7 +2445,7 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
* and the HVM restore program takes another.
* Also allow one typed refcount for xenheap pages, to match
* share_xen_page_with_guest(). */
- if ( !(shadow_mode_external(v->domain)
+ if ( !(shadow_mode_external(d)
&& (page->count_info & PGC_count_mask) <= 3
&& ((page->u.inuse.type_info & PGT_count_mask)
== !!is_xen_heap_page(page))) )
@@ -2452,7 +2456,7 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
}
}
- paging_unlock(v->domain);
+ paging_unlock(d);
/* We killed at least one mapping, so must flush TLBs. */
return 1;
@@ -2526,6 +2530,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
* (all != 0 implies fast == 0)
*/
{
+ struct domain *d = v->domain;
struct page_info *pg = mfn_to_page(gmfn);
mfn_t smfn;
unsigned char t;
@@ -2577,15 +2582,15 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
/* Although this is an externally visible function, we do not know
* whether the paging lock will be held when it is called (since it
* can be called via put_page_type when we clear a shadow l1e).*/
- paging_lock_recursive(v->domain);
+ paging_lock_recursive(d);
SHADOW_PRINTK("d=%d, v=%d, gmfn=%05lx\n",
- v->domain->domain_id, v->vcpu_id, mfn_x(gmfn));
+ d->domain_id, v->vcpu_id, mfn_x(gmfn));
/* Bail out now if the page is not shadowed */
if ( (pg->count_info & PGC_page_table) == 0 )
{
- paging_unlock(v->domain);
+ paging_unlock(d);
return;
}
@@ -2638,14 +2643,14 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
SHADOW_ERROR("can't find all shadows of mfn %05lx "
"(shadow_flags=%08x)\n",
mfn_x(gmfn), pg->shadow_flags);
- domain_crash(v->domain);
+ domain_crash(d);
}
/* Need to flush TLBs now, so that linear maps are safe next time we
* take a fault. */
- flush_tlb_mask(v->domain->domain_dirty_cpumask);
+ flush_tlb_mask(d->domain_dirty_cpumask);
- paging_unlock(v->domain);
+ paging_unlock(d);
}
static void
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 434df61..b538997 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -134,7 +134,7 @@ set_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
ASSERT(mfn_to_page(smfn)->u.sh.head);
/* 32-on-64 PV guests don't own their l4 pages so can't get_page them */
- if ( !is_pv_32on64_vcpu(v) || shadow_type != SH_type_l4_64_shadow )
+ if ( !is_pv_32on64_domain(d) || shadow_type != SH_type_l4_64_shadow )
{
res = get_page(mfn_to_page(gmfn), d);
ASSERT(res == 1);
@@ -157,13 +157,14 @@ static inline void
delete_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
/* Remove a shadow from the hash table */
{
+ struct domain *d = v->domain;
SHADOW_PRINTK("d=%d, v=%d, gmfn=%05lx, type=%08x, smfn=%05lx\n",
- v->domain->domain_id, v->vcpu_id,
+ d->domain_id, v->vcpu_id,
mfn_x(gmfn), shadow_type, mfn_x(smfn));
ASSERT(mfn_to_page(smfn)->u.sh.head);
shadow_hash_delete(v, mfn_x(gmfn), shadow_type, smfn);
/* 32-on-64 PV guests don't own their l4 pages; see set_shadow_status */
- if ( !is_pv_32on64_vcpu(v) || shadow_type != SH_type_l4_64_shadow )
+ if ( !is_pv_32on64_domain(d) || shadow_type != SH_type_l4_64_shadow )
put_page(mfn_to_page(gmfn));
}
@@ -887,6 +888,7 @@ static int shadow_set_l4e(struct vcpu *v,
shadow_l4e_t new_sl4e,
mfn_t sl4mfn)
{
+ struct domain *d = v->domain;
int flags = 0, ok;
shadow_l4e_t old_sl4e;
paddr_t paddr;
@@ -908,7 +910,7 @@ static int shadow_set_l4e(struct vcpu *v,
ok |= sh_pin(v, sl3mfn);
if ( !ok )
{
- domain_crash(v->domain);
+ domain_crash(d);
return SHADOW_SET_ERROR;
}
}
@@ -937,6 +939,7 @@ static int shadow_set_l3e(struct vcpu *v,
shadow_l3e_t new_sl3e,
mfn_t sl3mfn)
{
+ struct domain *d = v->domain;
int flags = 0;
shadow_l3e_t old_sl3e;
paddr_t paddr;
@@ -953,7 +956,7 @@ static int shadow_set_l3e(struct vcpu *v,
/* About to install a new reference */
if ( !sh_get_ref(v, shadow_l3e_get_mfn(new_sl3e), paddr) )
{
- domain_crash(v->domain);
+ domain_crash(d);
return SHADOW_SET_ERROR;
}
}
@@ -983,6 +986,7 @@ static int shadow_set_l2e(struct vcpu *v,
shadow_l2e_t new_sl2e,
mfn_t sl2mfn)
{
+ struct domain *d = v->domain;
int flags = 0;
shadow_l2e_t old_sl2e;
paddr_t paddr;
@@ -1014,7 +1018,7 @@ static int shadow_set_l2e(struct vcpu *v,
/* About to install a new reference */
if ( !sh_get_ref(v, sl1mfn, paddr) )
{
- domain_crash(v->domain);
+ domain_crash(d);
return SHADOW_SET_ERROR;
}
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
@@ -1495,7 +1499,8 @@ static void sh_install_xen_entries_in_l2h(struct vcpu *v, mfn_t sl2hmfn)
static mfn_t
sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
{
- mfn_t smfn = shadow_alloc(v->domain, shadow_type, mfn_x(gmfn));
+ struct domain *d = v->domain;
+ mfn_t smfn = shadow_alloc(d, shadow_type, mfn_x(gmfn));
SHADOW_DEBUG(MAKE_SHADOW, "(%05lx, %u)=>%05lx\n",
mfn_x(gmfn), shadow_type, mfn_x(smfn));
@@ -1506,7 +1511,7 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
#if GUEST_PAGING_LEVELS == 4
#if (SHADOW_OPTIMIZATIONS & SHOPT_LINUX_L3_TOPLEVEL)
if ( shadow_type == SH_type_l4_64_shadow &&
- unlikely(v->domain->arch.paging.shadow.opt_flags & SHOPT_LINUX_L3_TOPLEVEL) )
+ unlikely(d->arch.paging.shadow.opt_flags & SHOPT_LINUX_L3_TOPLEVEL) )
{
/* We're shadowing a new l4, but we've been assuming the guest uses
* only one l4 per vcpu and context switches using an l4 entry.
@@ -1517,22 +1522,22 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
struct page_info *sp, *t;
struct vcpu *v2;
int l4count = 0, vcpus = 0;
- page_list_for_each(sp, &v->domain->arch.paging.shadow.pinned_shadows)
+ page_list_for_each(sp, &d->arch.paging.shadow.pinned_shadows)
{
if ( sp->u.sh.type == SH_type_l4_64_shadow )
l4count++;
}
- for_each_vcpu ( v->domain, v2 )
+ for_each_vcpu ( d, v2 )
vcpus++;
if ( l4count > 2 * vcpus )
{
/* Unpin all the pinned l3 tables, and don't pin any more. */
- page_list_for_each_safe(sp, t, &v->domain->arch.paging.shadow.pinned_shadows)
+ page_list_for_each_safe(sp, t, &d->arch.paging.shadow.pinned_shadows)
{
if ( sp->u.sh.type == SH_type_l3_64_shadow )
sh_unpin(v, page_to_mfn(sp));
}
- v->domain->arch.paging.shadow.opt_flags &= ~SHOPT_LINUX_L3_TOPLEVEL;
+ d->arch.paging.shadow.opt_flags &= ~SHOPT_LINUX_L3_TOPLEVEL;
sh_reset_l3_up_pointers(v);
}
}
@@ -1540,7 +1545,7 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
#endif
// Create the Xen mappings...
- if ( !shadow_mode_external(v->domain) )
+ if ( !shadow_mode_external(d) )
{
switch (shadow_type)
{
@@ -1566,8 +1571,8 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
static mfn_t
make_fl1_shadow(struct vcpu *v, gfn_t gfn)
{
- mfn_t smfn = shadow_alloc(v->domain, SH_type_fl1_shadow,
- (unsigned long) gfn_x(gfn));
+ struct domain *d = v->domain;
+ mfn_t smfn = shadow_alloc(d, SH_type_fl1_shadow, gfn_x(gfn));
SHADOW_DEBUG(MAKE_SHADOW, "(%" SH_PRI_gfn ")=>%" PRI_mfn "\n",
gfn_x(gfn), mfn_x(smfn));
@@ -1875,6 +1880,7 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
#if GUEST_PAGING_LEVELS >= 4
void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
{
+ struct domain *d = v->domain;
shadow_l4e_t *sl4e;
struct page_info *sp = mfn_to_page(smfn);
u32 t = sp->u.sh.type;
@@ -1891,7 +1897,7 @@ void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
shadow_demote(v, gmfn, t);
/* Decrement refcounts of all the old entries */
sl4mfn = smfn;
- SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, v->domain, {
+ SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, d, {
if ( shadow_l4e_get_flags(*sl4e) & _PAGE_PRESENT )
{
sh_put_ref(v, shadow_l4e_get_mfn(*sl4e),
@@ -1901,11 +1907,12 @@ void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
});
/* Put the memory back in the pool */
- shadow_free(v->domain, smfn);
+ shadow_free(d, smfn);
}
void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
{
+ struct domain *d = v->domain;
shadow_l3e_t *sl3e;
struct page_info *sp = mfn_to_page(smfn);
u32 t = sp->u.sh.type;
@@ -1931,13 +1938,14 @@ void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
});
/* Put the memory back in the pool */
- shadow_free(v->domain, smfn);
+ shadow_free(d, smfn);
}
#endif /* GUEST_PAGING_LEVELS >= 4 */
void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
{
+ struct domain *d = v->domain;
shadow_l2e_t *sl2e;
struct page_info *sp = mfn_to_page(smfn);
u32 t = sp->u.sh.type;
@@ -1960,7 +1968,7 @@ void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
/* Decrement refcounts of all the old entries */
sl2mfn = smfn;
- SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, v->domain, {
+ SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( shadow_l2e_get_flags(*sl2e) & _PAGE_PRESENT )
sh_put_ref(v, shadow_l2e_get_mfn(*sl2e),
(((paddr_t)mfn_x(sl2mfn)) << PAGE_SHIFT)
@@ -1968,7 +1976,7 @@ void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
});
/* Put the memory back in the pool */
- shadow_free(v->domain, smfn);
+ shadow_free(d, smfn);
}
void sh_destroy_l1_shadow(struct vcpu *v, mfn_t smfn)
@@ -2010,7 +2018,7 @@ void sh_destroy_l1_shadow(struct vcpu *v, mfn_t smfn)
}
/* Put the memory back in the pool */
- shadow_free(v->domain, smfn);
+ shadow_free(d, smfn);
}
#if SHADOW_PAGING_LEVELS == GUEST_PAGING_LEVELS
@@ -2072,8 +2080,9 @@ void sh_destroy_monitor_table(struct vcpu *v, mfn_t mmfn)
void sh_unhook_32b_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
{
+ struct domain *d = v->domain;
shadow_l2e_t *sl2e;
- SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, v->domain, {
+ SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( !user_only || (sl2e->l2 & _PAGE_USER) )
(void) shadow_set_l2e(v, sl2e, shadow_l2e_empty(), sl2mfn);
});
@@ -2084,8 +2093,9 @@ void sh_unhook_32b_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
void sh_unhook_pae_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
/* Walk a PAE l2 shadow, unhooking entries from all the subshadows */
{
+ struct domain *d = v->domain;
shadow_l2e_t *sl2e;
- SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, v->domain, {
+ SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( !user_only || (sl2e->l2 & _PAGE_USER) )
(void) shadow_set_l2e(v, sl2e, shadow_l2e_empty(), sl2mfn);
});
@@ -2095,8 +2105,9 @@ void sh_unhook_pae_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
void sh_unhook_64b_mappings(struct vcpu *v, mfn_t sl4mfn, int user_only)
{
+ struct domain *d = v->domain;
shadow_l4e_t *sl4e;
- SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, v->domain, {
+ SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, d, {
if ( !user_only || (sl4e->l4 & _PAGE_USER) )
(void) shadow_set_l4e(v, sl4e, shadow_l4e_empty(), sl4mfn);
});
@@ -2176,6 +2187,7 @@ static int validate_gl4e(struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
{
+ struct domain *d = v->domain;
shadow_l3e_t new_sl3e;
guest_l3e_t new_gl3e = *(guest_l3e_t *)new_ge;
shadow_l3e_t *sl3p = se;
@@ -2188,7 +2200,7 @@ static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
if ( guest_l3e_get_flags(new_gl3e) & _PAGE_PRESENT )
{
gfn_t gl2gfn = guest_l3e_get_gfn(new_gl3e);
- mfn_t gl2mfn = get_gfn_query_unlocked(v->domain, gfn_x(gl2gfn), &p2mt);
+ mfn_t gl2mfn = get_gfn_query_unlocked(d, gfn_x(gl2gfn), &p2mt);
if ( p2m_is_ram(p2mt) )
sl2mfn = get_shadow_status(v, gl2mfn, SH_type_l2_shadow);
else if ( p2mt != p2m_populate_on_demand )
@@ -2208,6 +2220,7 @@ static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
{
+ struct domain *d = v->domain;
shadow_l2e_t new_sl2e;
guest_l2e_t new_gl2e = *(guest_l2e_t *)new_ge;
shadow_l2e_t *sl2p = se;
@@ -2237,8 +2250,7 @@ static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
}
else
{
- mfn_t gl1mfn = get_gfn_query_unlocked(v->domain, gfn_x(gl1gfn),
- &p2mt);
+ mfn_t gl1mfn = get_gfn_query_unlocked(d, gfn_x(gl1gfn), &p2mt);
if ( p2m_is_ram(p2mt) )
sl1mfn = get_shadow_status(v, gl1mfn, SH_type_l1_shadow);
else if ( p2mt != p2m_populate_on_demand )
@@ -2254,6 +2266,7 @@ static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
{
+ struct domain *d = v->domain;
shadow_l1e_t new_sl1e;
guest_l1e_t new_gl1e = *(guest_l1e_t *)new_ge;
shadow_l1e_t *sl1p = se;
@@ -2268,7 +2281,7 @@ static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
perfc_incr(shadow_validate_gl1e_calls);
gfn = guest_l1e_get_gfn(new_gl1e);
- gmfn = get_gfn_query_unlocked(v->domain, gfn_x(gfn), &p2mt);
+ gmfn = get_gfn_query_unlocked(d, gfn_x(gfn), &p2mt);
l1e_propagate_from_guest(v, new_gl1e, gmfn, &new_sl1e, ft_prefetch, p2mt);
result |= shadow_set_l1e(v, sl1p, new_sl1e, p2mt, sl1mfn);
@@ -2302,6 +2315,7 @@ static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
* *not* the one that is causing it to be resynced. */
void sh_resync_l1(struct vcpu *v, mfn_t gl1mfn, mfn_t snpmfn)
{
+ struct domain *d = v->domain;
mfn_t sl1mfn;
shadow_l1e_t *sl1p;
guest_l1e_t *gl1p, *gp, *snp;
@@ -2328,7 +2342,7 @@ void sh_resync_l1(struct vcpu *v, mfn_t gl1mfn, mfn_t snpmfn)
shadow_l1e_t nsl1e;
gfn = guest_l1e_get_gfn(gl1e);
- gmfn = get_gfn_query_unlocked(v->domain, gfn_x(gfn), &p2mt);
+ gmfn = get_gfn_query_unlocked(d, gfn_x(gfn), &p2mt);
l1e_propagate_from_guest(v, gl1e, gmfn, &nsl1e, ft_prefetch, p2mt);
rc |= shadow_set_l1e(v, sl1p, nsl1e, p2mt, sl1mfn);
*snpl1p = gl1e;
@@ -2539,6 +2553,7 @@ sh_map_and_validate_gl1e(struct vcpu *v, mfn_t gl1mfn,
static inline void check_for_early_unshadow(struct vcpu *v, mfn_t gmfn)
{
#if SHADOW_OPTIMIZATIONS & SHOPT_EARLY_UNSHADOW
+ struct domain *d = v->domain;
/* If the domain has never made a "dying" op, use the two-writes
* heuristic; otherwise, unshadow as soon as we write a zero for a dying
* process.
@@ -2546,10 +2561,10 @@ static inline void check_for_early_unshadow(struct vcpu *v, mfn_t gmfn)
* Don't bother trying to unshadow if it's not a PT, or if it's > l1.
*/
if ( ( v->arch.paging.shadow.pagetable_dying
- || ( !v->domain->arch.paging.shadow.pagetable_dying_op
+ || ( !d->arch.paging.shadow.pagetable_dying_op
&& v->arch.paging.shadow.last_emulated_mfn_for_unshadow == mfn_x(gmfn) ) )
&& sh_mfn_is_a_page_table(gmfn)
- && (!v->domain->arch.paging.shadow.pagetable_dying_op ||
+ && (!d->arch.paging.shadow.pagetable_dying_op ||
!(mfn_to_page(gmfn)->shadow_flags
& (SHF_L2_32|SHF_L2_PAE|SHF_L2H_PAE|SHF_L4_64))) )
{
@@ -2585,6 +2600,7 @@ static inline void reset_early_unshadow(struct vcpu *v)
static void sh_prefetch(struct vcpu *v, walk_t *gw,
shadow_l1e_t *ptr_sl1e, mfn_t sl1mfn)
{
+ struct domain *d = v->domain;
int i, dist;
gfn_t gfn;
mfn_t gmfn;
@@ -2651,7 +2667,7 @@ static void sh_prefetch(struct vcpu *v, walk_t *gw,
/* Look at the gfn that the l1e is pointing at */
gfn = guest_l1e_get_gfn(gl1e);
- gmfn = get_gfn_query_unlocked(v->domain, gfn_x(gfn), &p2mt);
+ gmfn = get_gfn_query_unlocked(d, gfn_x(gfn), &p2mt);
/* Propagate the entry. */
l1e_propagate_from_guest(v, gl1e, gmfn, &sl1e, ft_prefetch, p2mt);
@@ -3532,6 +3548,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Check to see if the SL1 is out of sync. */
{
+ struct domain *d = v->domain;
mfn_t gl1mfn = backpointer(mfn_to_page(sl1mfn));
struct page_info *pg = mfn_to_page(gl1mfn);
if ( mfn_valid(gl1mfn)
@@ -3539,7 +3556,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
{
/* The test above may give false positives, since we don't
* hold the paging lock yet. Check again with the lock held. */
- paging_lock(v->domain);
+ paging_lock(d);
/* This must still be a copy-from-user because we didn't
* have the paging lock last time we checked, and the
@@ -3551,13 +3568,13 @@ sh_invlpg(struct vcpu *v, unsigned long va)
sizeof (sl2e)) != 0 )
{
perfc_incr(shadow_invlpg_fault);
- paging_unlock(v->domain);
+ paging_unlock(d);
return 0;
}
if ( !(shadow_l2e_get_flags(sl2e) & _PAGE_PRESENT) )
{
- paging_unlock(v->domain);
+ paging_unlock(d);
return 0;
}
@@ -3574,7 +3591,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
(void) shadow_set_l1e(v, sl1, shadow_l1e_empty(),
p2m_invalid, sl1mfn);
}
- paging_unlock(v->domain);
+ paging_unlock(d);
/* Need the invlpg, to pick up the disappeareance of the sl1e */
return 1;
}
@@ -3850,7 +3867,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
if ( sh_pin(v, smfn) == 0 )
{
SHADOW_ERROR("can't pin %#lx as toplevel shadow\n", mfn_x(smfn));
- domain_crash(v->domain);
+ domain_crash(d);
}
/* Take a ref to this page: it will be released in sh_detach_old_tables()
@@ -3858,7 +3875,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
if ( !sh_get_ref(v, smfn, 0) )
{
SHADOW_ERROR("can't install %#lx as toplevel shadow\n", mfn_x(smfn));
- domain_crash(v->domain);
+ domain_crash(d);
}
new_entry = pagetable_from_mfn(smfn);
@@ -3879,7 +3896,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
if ( !mfn_to_page(old_smfn)->u.sh.pinned && !sh_pin(v, old_smfn) )
{
SHADOW_ERROR("can't re-pin %#lx\n", mfn_x(old_smfn));
- domain_crash(v->domain);
+ domain_crash(d);
}
sh_put_ref(v, old_smfn, 0);
}
@@ -4354,11 +4371,12 @@ void sh_clear_shadow_entry(struct vcpu *v, void *ep, mfn_t smfn)
int sh_remove_l1_shadow(struct vcpu *v, mfn_t sl2mfn, mfn_t sl1mfn)
/* Remove all mappings of this l1 shadow from this l2 shadow */
{
+ struct domain *d = v->domain;
shadow_l2e_t *sl2e;
int done = 0;
int flags;
- SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, done, v->domain,
+ SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, done, d,
{
flags = shadow_l2e_get_flags(*sl2e);
if ( (flags & _PAGE_PRESENT)
@@ -4399,11 +4417,12 @@ int sh_remove_l2_shadow(struct vcpu *v, mfn_t sl3mfn, mfn_t sl2mfn)
int sh_remove_l3_shadow(struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn)
/* Remove all mappings of this l3 shadow from this l4 shadow */
{
+ struct domain *d = v->domain;
shadow_l4e_t *sl4e;
int done = 0;
int flags;
- SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, done, v->domain,
+ SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, done, d,
{
flags = shadow_l4e_get_flags(*sl4e);
if ( (flags & _PAGE_PRESENT)
@@ -4427,6 +4446,7 @@ int sh_remove_l3_shadow(struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn)
#if GUEST_PAGING_LEVELS == 3
static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
{
+ struct domain *d = v->domain;
int i = 0;
int flush = 0;
int fast_path = 0;
@@ -4444,7 +4464,7 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
fast_path = 1;
l3gfn = gpa >> PAGE_SHIFT;
- l3mfn = get_gfn_query(v->domain, _gfn(l3gfn), &p2mt);
+ l3mfn = get_gfn_query(d, _gfn(l3gfn), &p2mt);
if ( !mfn_valid(l3mfn) || !p2m_is_ram(p2mt) )
{
printk(XENLOG_DEBUG "sh_pagetable_dying: gpa not valid %"PRIpaddr"\n",
@@ -4452,7 +4472,7 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
goto out_put_gfn;
}
- paging_lock(v->domain);
+ paging_lock(d);
if ( !fast_path )
{
@@ -4475,7 +4495,7 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
/* retrieving the l2s */
gl2a = guest_l3e_get_paddr(gl3e[i]);
gfn = gl2a >> PAGE_SHIFT;
- gmfn = get_gfn_query_unlocked(v->domain, gfn, &p2mt);
+ gmfn = get_gfn_query_unlocked(d, gfn, &p2mt);
smfn = shadow_hash_lookup(v, mfn_x(gmfn), SH_type_l2_pae_shadow);
}
@@ -4488,29 +4508,30 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
}
}
if ( flush )
- flush_tlb_mask(v->domain->domain_dirty_cpumask);
+ flush_tlb_mask(d->domain_dirty_cpumask);
/* Remember that we've seen the guest use this interface, so we
* can rely on it using it in future, instead of guessing at
* when processes are being torn down. */
- v->domain->arch.paging.shadow.pagetable_dying_op = 1;
+ d->arch.paging.shadow.pagetable_dying_op = 1;
v->arch.paging.shadow.pagetable_dying = 1;
if ( !fast_path )
unmap_domain_page(gl3pa);
- paging_unlock(v->domain);
+ paging_unlock(d);
out_put_gfn:
- put_gfn(v->domain, l3gfn);
+ put_gfn(d, l3gfn);
}
#else
static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
{
+ struct domain *d = v->domain;
mfn_t smfn, gmfn;
p2m_type_t p2mt;
- gmfn = get_gfn_query(v->domain, _gfn(gpa >> PAGE_SHIFT), &p2mt);
- paging_lock(v->domain);
+ gmfn = get_gfn_query(d, _gfn(gpa >> PAGE_SHIFT), &p2mt);
+ paging_lock(d);
#if GUEST_PAGING_LEVELS == 2
smfn = shadow_hash_lookup(v, mfn_x(gmfn), SH_type_l2_32_shadow);
@@ -4523,18 +4544,18 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
mfn_to_page(gmfn)->shadow_flags |= SHF_pagetable_dying;
shadow_unhook_mappings(v, smfn, 1/* user pages only */);
/* Now flush the TLB: we removed toplevel mappings. */
- flush_tlb_mask(v->domain->domain_dirty_cpumask);
+ flush_tlb_mask(d->domain_dirty_cpumask);
}
/* Remember that we've seen the guest use this interface, so we
* can rely on it using it in future, instead of guessing at
* when processes are being torn down. */
- v->domain->arch.paging.shadow.pagetable_dying_op = 1;
+ d->arch.paging.shadow.pagetable_dying_op = 1;
v->arch.paging.shadow.pagetable_dying = 1;
- paging_unlock(v->domain);
- put_gfn(v->domain, gpa >> PAGE_SHIFT);
+ paging_unlock(d);
+ put_gfn(d, gpa >> PAGE_SHIFT);
}
#endif
@@ -4987,6 +5008,7 @@ int sh_audit_fl1_table(struct vcpu *v, mfn_t sl1mfn, mfn_t x)
int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
{
+ struct domain *d = v->domain;
guest_l2e_t *gl2e, *gp;
shadow_l2e_t *sl2e;
mfn_t mfn, gmfn, gl2mfn;
@@ -5006,7 +5028,7 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
#endif
gl2e = gp = sh_map_domain_page(gl2mfn);
- SHADOW_FOREACH_L2E(sl2mfn, sl2e, &gl2e, done, v->domain, {
+ SHADOW_FOREACH_L2E(sl2mfn, sl2e, &gl2e, done, d, {
s = sh_audit_flags(v, 2, guest_l2e_get_flags(*gl2e),
shadow_l2e_get_flags(*sl2e));
@@ -5019,7 +5041,7 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
gmfn = (guest_l2e_get_flags(*gl2e) & _PAGE_PSE)
? get_fl1_shadow_status(v, gfn)
: get_shadow_status(v,
- get_gfn_query_unlocked(v->domain, gfn_x(gfn),
+ get_gfn_query_unlocked(d, gfn_x(gfn),
&p2mt), SH_type_l1_shadow);
if ( mfn_x(gmfn) != mfn_x(mfn) )
AUDIT_FAIL(2, "bad translation: gfn %" SH_PRI_gfn
@@ -5027,7 +5049,7 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
" --> %" PRI_mfn " != mfn %" PRI_mfn,
gfn_x(gfn),
(guest_l2e_get_flags(*gl2e) & _PAGE_PSE) ? 0
- : mfn_x(get_gfn_query_unlocked(v->domain,
+ : mfn_x(get_gfn_query_unlocked(d,
gfn_x(gfn), &p2mt)), mfn_x(gmfn), mfn_x(mfn));
}
});
@@ -5038,6 +5060,7 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
#if GUEST_PAGING_LEVELS >= 4
int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
{
+ struct domain *d = v->domain;
guest_l3e_t *gl3e, *gp;
shadow_l3e_t *sl3e;
mfn_t mfn, gmfn, gl3mfn;
@@ -5068,10 +5091,10 @@ int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
gfn = guest_l3e_get_gfn(*gl3e);
mfn = shadow_l3e_get_mfn(*sl3e);
gmfn = get_shadow_status(v, get_gfn_query_unlocked(
- v->domain, gfn_x(gfn), &p2mt),
+ d, gfn_x(gfn), &p2mt),
((GUEST_PAGING_LEVELS == 3 ||
is_pv_32on64_vcpu(v))
- && !shadow_mode_external(v->domain)
+ && !shadow_mode_external(d)
&& (guest_index(gl3e) % 4) == 3)
? SH_type_l2h_shadow
: SH_type_l2_shadow);
@@ -5087,6 +5110,7 @@ int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
{
+ struct domain *d = v->domain;
guest_l4e_t *gl4e, *gp;
shadow_l4e_t *sl4e;
mfn_t mfn, gmfn, gl4mfn;
@@ -5106,7 +5130,7 @@ int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
#endif
gl4e = gp = sh_map_domain_page(gl4mfn);
- SHADOW_FOREACH_L4E(sl4mfn, sl4e, &gl4e, done, v->domain,
+ SHADOW_FOREACH_L4E(sl4mfn, sl4e, &gl4e, done, d,
{
s = sh_audit_flags(v, 4, guest_l4e_get_flags(*gl4e),
shadow_l4e_get_flags(*sl4e));
@@ -5117,7 +5141,7 @@ int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
gfn = guest_l4e_get_gfn(*gl4e);
mfn = shadow_l4e_get_mfn(*sl4e);
gmfn = get_shadow_status(v, get_gfn_query_unlocked(
- v->domain, gfn_x(gfn), &p2mt),
+ d, gfn_x(gfn), &p2mt),
SH_type_l3_shadow);
if ( mfn_x(gmfn) != mfn_x(mfn) )
AUDIT_FAIL(4, "bad translation: gfn %" SH_PRI_gfn
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index 4b69626..baa271c 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -646,6 +646,7 @@ prev_pinned_shadow(const struct page_info *page,
* Returns 0 for failure, 1 for success. */
static inline int sh_pin(struct vcpu *v, mfn_t smfn)
{
+ struct domain *d = v->domain;
struct page_info *sp[4];
struct page_list_head *pin_list;
unsigned int i, pages;
@@ -658,7 +659,7 @@ static inline int sh_pin(struct vcpu *v, mfn_t smfn)
ASSERT(sh_type_is_pinnable(v, sp[0]->u.sh.type));
ASSERT(sp[0]->u.sh.head);
- pin_list = &v->domain->arch.paging.shadow.pinned_shadows;
+ pin_list = &d->arch.paging.shadow.pinned_shadows;
if ( already_pinned && sp[0] == page_list_first(pin_list) )
return 1;
@@ -695,6 +696,7 @@ static inline int sh_pin(struct vcpu *v, mfn_t smfn)
* of pinned shadows, and release the extra ref. */
static inline void sh_unpin(struct vcpu *v, mfn_t smfn)
{
+ struct domain *d = v->domain;
struct page_list_head tmp_list, *pin_list;
struct page_info *sp, *next;
unsigned int i, head_type;
@@ -711,7 +713,7 @@ static inline void sh_unpin(struct vcpu *v, mfn_t smfn)
/* Cut the sub-list out of the list of pinned shadows,
* stitching it back into a list fragment of its own. */
- pin_list = &v->domain->arch.paging.shadow.pinned_shadows;
+ pin_list = &d->arch.paging.shadow.pinned_shadows;
INIT_PAGE_LIST_HEAD(&tmp_list);
for ( i = 0; i < shadow_size(head_type); i++ )
{
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 04/20] x86/shadow: Only apply shadow heuristics when in guest context
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (2 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 03/20] x86/shadow: Introduce 'd' pointers and clean up use of 'v->domain' Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-19 13:10 ` Tim Deegan
2015-02-12 17:16 ` [PATCH 05/20] x86/shadow: Alter shadow_hash_{lookup, insert, delete}() to take a domain Andrew Cooper
` (17 subsequent siblings)
21 siblings, 1 reply; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
It is incorrect to be applying these heuristics because of toolstack actions.
As the vcpu parameters are to be replaced with domain parameters, guest
context is identified by using current->domain.
Note that the majority of the heuristics in sh_remove_write_access() were
already restricted to guest context, but they are updated to use 'curr' for
clarity.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 21 +++++++++++++--------
xen/arch/x86/mm/shadow/multi.c | 13 +++++++++----
2 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 3b5ef19..26dab30 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -2170,6 +2170,9 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
;
struct domain *d = v->domain;
struct page_info *pg = mfn_to_page(gmfn);
+#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
+ struct vcpu *curr = current;
+#endif
ASSERT(paging_locked_by_me(d));
@@ -2205,7 +2208,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
}
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
- if ( v == current )
+ if ( curr->domain == d )
{
unsigned long gfn;
/* Heuristic: there is likely to be only one writeable mapping,
@@ -2213,7 +2216,8 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
* in the guest's linear map (on non-HIGHPTE linux and windows)*/
#define GUESS(_a, _h) do { \
- if ( v->arch.paging.mode->shadow.guess_wrmap(v, (_a), gmfn) ) \
+ if ( curr->arch.paging.mode->shadow.guess_wrmap( \
+ curr, (_a), gmfn) ) \
perfc_incr(shadow_writeable_h_ ## _h); \
if ( (pg->u.inuse.type_info & PGT_count_mask) == 0 ) \
{ \
@@ -2222,7 +2226,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
} \
} while (0)
- if ( v->arch.paging.mode->guest_levels == 2 )
+ if ( curr->arch.paging.mode->guest_levels == 2 )
{
if ( level == 1 )
/* 32bit non-PAE w2k3: linear map at 0xC0000000 */
@@ -2237,7 +2241,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
GUESS(0xBFC00000UL
+ ((fault_addr & VADDR_MASK) >> 10), 6);
}
- else if ( v->arch.paging.mode->guest_levels == 3 )
+ else if ( curr->arch.paging.mode->guest_levels == 3 )
{
/* 32bit PAE w2k3: linear map at 0xC0000000 */
switch ( level )
@@ -2259,7 +2263,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
+ ((fault_addr & VADDR_MASK) >> 18), 6); break;
}
}
- else if ( v->arch.paging.mode->guest_levels == 4 )
+ else if ( curr->arch.paging.mode->guest_levels == 4 )
{
/* 64bit w2k3: linear map at 0xfffff68000000000 */
switch ( level )
@@ -2312,14 +2316,15 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
* the writeable mapping by looking at the same MFN where the last
* brute-force search succeeded. */
- if ( v->arch.paging.shadow.last_writeable_pte_smfn != 0 )
+ if ( (curr->domain == d) &&
+ (curr->arch.paging.shadow.last_writeable_pte_smfn != 0) )
{
unsigned long old_count = (pg->u.inuse.type_info & PGT_count_mask);
- mfn_t last_smfn = _mfn(v->arch.paging.shadow.last_writeable_pte_smfn);
+ mfn_t last_smfn = _mfn(curr->arch.paging.shadow.last_writeable_pte_smfn);
int shtype = mfn_to_page(last_smfn)->u.sh.type;
if ( callbacks[shtype] )
- callbacks[shtype](v, last_smfn, gmfn);
+ callbacks[shtype](curr, last_smfn, gmfn);
if ( (pg->u.inuse.type_info & PGT_count_mask) != old_count )
perfc_incr(shadow_writeable_h_5);
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index b538997..f532bff 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -4185,6 +4185,8 @@ sh_update_cr3(struct vcpu *v, int do_locking)
int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
mfn_t smfn, unsigned long off)
{
+ struct domain *d = v->domain;
+ struct vcpu *curr = current;
int r;
shadow_l1e_t *sl1p, sl1e;
struct page_info *sp;
@@ -4193,9 +4195,9 @@ int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
ASSERT(mfn_valid(smfn));
/* Remember if we've been told that this process is being torn down */
- v->arch.paging.shadow.pagetable_dying
- = !!(mfn_to_page(gmfn)->shadow_flags & SHF_pagetable_dying);
-
+ if ( curr->domain == d )
+ curr->arch.paging.shadow.pagetable_dying
+ = !!(mfn_to_page(gmfn)->shadow_flags & SHF_pagetable_dying);
sp = mfn_to_page(smfn);
@@ -4290,6 +4292,8 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
int done = 0;
int flags;
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
+ struct domain *d = v->domain;
+ struct vcpu *curr = current;
mfn_t base_sl1mfn = sl1mfn; /* Because sl1mfn changes in the foreach */
#endif
@@ -4304,7 +4308,8 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
(void) shadow_set_l1e(v, sl1e, ro_sl1e, p2m_ram_rw, sl1mfn);
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
/* Remember the last shadow that we shot a writeable mapping in */
- v->arch.paging.shadow.last_writeable_pte_smfn = mfn_x(base_sl1mfn);
+ if ( curr->domain == d )
+ curr->arch.paging.shadow.last_writeable_pte_smfn = mfn_x(base_sl1mfn);
#endif
if ( (mfn_to_page(readonly_mfn)->u.inuse.type_info
& PGT_count_mask) == 0 )
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH 04/20] x86/shadow: Only apply shadow heuristics when in guest context
2015-02-12 17:16 ` [PATCH 04/20] x86/shadow: Only apply shadow heuristics when in guest context Andrew Cooper
@ 2015-02-19 13:10 ` Tim Deegan
2015-02-19 14:18 ` Andrew Cooper
2015-02-19 15:33 ` [PATCH v2 04/20] x86/shadow: Change the gating of shadow heuristics Andrew Cooper
0 siblings, 2 replies; 29+ messages in thread
From: Tim Deegan @ 2015-02-19 13:10 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Jan Beulich, Xen-devel
At 17:16 +0000 on 12 Feb (1423757763), Andrew Cooper wrote:
> It is incorrect to be applying these heuristics because of toolstack actions.
It might be incorrect if we did, but we don't. :P
The guess_wrmap() heuristics should only be called on v == current()
because they use the shadow linear map, but this patch actually
_relaxes_ that constraint so that they could run on other vcpus of the
same guest (harmlessly).
The second heuristic uses a stored smfn of a PTE, and uses the correct
callback for the shadow page's type, so is safe to call from any vcpu
so long as the domain's shadow lock is held. The check you add here
is needed _only_ because you are changing from using 'v' to using
'curr' for the test.
The changes you've made in multi.c are again only needed because you
are s/v/curr/ in the same patch, and again if anything are relaxing
the checks. Luckily they don't cover any mode-specific operations, so
AFAICT the code is still safe after all these changes, and at least no
worse than existing paths that use vcpu[0].
So this patch is safe, but it's not a bugfix, and I think the
changeset description needs to be rewritten. What you're actually
doing is removing uses of the vcpu parameter in these functions,
replacing them with uses of current. The changes to gating are a
side-effect.
With that change, Reviewed-by: Tim Deegan <tim@xen.org>
Cheers,
Tim.
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH 04/20] x86/shadow: Only apply shadow heuristics when in guest context
2015-02-19 13:10 ` Tim Deegan
@ 2015-02-19 14:18 ` Andrew Cooper
2015-02-19 15:33 ` [PATCH v2 04/20] x86/shadow: Change the gating of shadow heuristics Andrew Cooper
1 sibling, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-19 14:18 UTC (permalink / raw)
To: Tim Deegan; +Cc: Jan Beulich, Xen-devel
On 19/02/15 13:10, Tim Deegan wrote:
> At 17:16 +0000 on 12 Feb (1423757763), Andrew Cooper wrote:
>> It is incorrect to be applying these heuristics because of toolstack actions.
> It might be incorrect if we did, but we don't. :P
>
> The guess_wrmap() heuristics should only be called on v == current()
> because they use the shadow linear map, but this patch actually
> _relaxes_ that constraint so that they could run on other vcpus of the
> same guest (harmlessly).
>
> The second heuristic uses a stored smfn of a PTE, and uses the correct
> callback for the shadow page's type, so is safe to call from any vcpu
> so long as the domain's shadow lock is held. The check you add here
> is needed _only_ because you are changing from using 'v' to using
> 'curr' for the test.
>
> The changes you've made in multi.c are again only needed because you
> are s/v/curr/ in the same patch, and again if anything are relaxing
> the checks. Luckily they don't cover any mode-specific operations, so
> AFAICT the code is still safe after all these changes, and at least no
> worse than existing paths that use vcpu[0].
>
> So this patch is safe, but it's not a bugfix, and I think the
> changeset description needs to be rewritten. What you're actually
> doing is removing uses of the vcpu parameter in these functions,
> replacing them with uses of current. The changes to gating are a
> side-effect.
>
> With that change, Reviewed-by: Tim Deegan <tim@xen.org>
Thanks for taking the time to review this. I will rewrite the commit
message and submit a v2.
~Andrew
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v2 04/20] x86/shadow: Change the gating of shadow heuristics
2015-02-19 13:10 ` Tim Deegan
2015-02-19 14:18 ` Andrew Cooper
@ 2015-02-19 15:33 ` Andrew Cooper
1 sibling, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-19 15:33 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Each of these functions will have their vcpu paramters replaced with domain
parameters because they are part of domain-generic rather than vcpu specific
codepaths, which means that the use of 'v' will have to change. 'current' can
be used to obtain a vcpu when in an appropriate context.
The 'curr->domain == d' test is less restrictive than 'v == current'. The end
result is still safe as the code still only runs in the context of the correct
domain, but is now valid to run in cases where previously 'v' was some other
vcpu in the same domin.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Tim Deegan <tim@xen.org>
CC: Jan Beulich <JBeulich@suse.com>
---
xen/arch/x86/mm/shadow/common.c | 21 +++++++++++++--------
xen/arch/x86/mm/shadow/multi.c | 13 +++++++++----
2 files changed, 22 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 3b5ef19..26dab30 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -2170,6 +2170,9 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
;
struct domain *d = v->domain;
struct page_info *pg = mfn_to_page(gmfn);
+#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
+ struct vcpu *curr = current;
+#endif
ASSERT(paging_locked_by_me(d));
@@ -2205,7 +2208,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
}
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
- if ( v == current )
+ if ( curr->domain == d )
{
unsigned long gfn;
/* Heuristic: there is likely to be only one writeable mapping,
@@ -2213,7 +2216,8 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
* in the guest's linear map (on non-HIGHPTE linux and windows)*/
#define GUESS(_a, _h) do { \
- if ( v->arch.paging.mode->shadow.guess_wrmap(v, (_a), gmfn) ) \
+ if ( curr->arch.paging.mode->shadow.guess_wrmap( \
+ curr, (_a), gmfn) ) \
perfc_incr(shadow_writeable_h_ ## _h); \
if ( (pg->u.inuse.type_info & PGT_count_mask) == 0 ) \
{ \
@@ -2222,7 +2226,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
} \
} while (0)
- if ( v->arch.paging.mode->guest_levels == 2 )
+ if ( curr->arch.paging.mode->guest_levels == 2 )
{
if ( level == 1 )
/* 32bit non-PAE w2k3: linear map at 0xC0000000 */
@@ -2237,7 +2241,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
GUESS(0xBFC00000UL
+ ((fault_addr & VADDR_MASK) >> 10), 6);
}
- else if ( v->arch.paging.mode->guest_levels == 3 )
+ else if ( curr->arch.paging.mode->guest_levels == 3 )
{
/* 32bit PAE w2k3: linear map at 0xC0000000 */
switch ( level )
@@ -2259,7 +2263,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
+ ((fault_addr & VADDR_MASK) >> 18), 6); break;
}
}
- else if ( v->arch.paging.mode->guest_levels == 4 )
+ else if ( curr->arch.paging.mode->guest_levels == 4 )
{
/* 64bit w2k3: linear map at 0xfffff68000000000 */
switch ( level )
@@ -2312,14 +2316,15 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
* the writeable mapping by looking at the same MFN where the last
* brute-force search succeeded. */
- if ( v->arch.paging.shadow.last_writeable_pte_smfn != 0 )
+ if ( (curr->domain == d) &&
+ (curr->arch.paging.shadow.last_writeable_pte_smfn != 0) )
{
unsigned long old_count = (pg->u.inuse.type_info & PGT_count_mask);
- mfn_t last_smfn = _mfn(v->arch.paging.shadow.last_writeable_pte_smfn);
+ mfn_t last_smfn = _mfn(curr->arch.paging.shadow.last_writeable_pte_smfn);
int shtype = mfn_to_page(last_smfn)->u.sh.type;
if ( callbacks[shtype] )
- callbacks[shtype](v, last_smfn, gmfn);
+ callbacks[shtype](curr, last_smfn, gmfn);
if ( (pg->u.inuse.type_info & PGT_count_mask) != old_count )
perfc_incr(shadow_writeable_h_5);
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index b538997..f532bff 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -4185,6 +4185,8 @@ static int sh_page_fault(struct vcpu *v,
int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
mfn_t smfn, unsigned long off)
{
+ struct domain *d = v->domain;
+ struct vcpu *curr = current;
int r;
shadow_l1e_t *sl1p, sl1e;
struct page_info *sp;
@@ -4193,9 +4195,9 @@ int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
ASSERT(mfn_valid(smfn));
/* Remember if we've been told that this process is being torn down */
- v->arch.paging.shadow.pagetable_dying
- = !!(mfn_to_page(gmfn)->shadow_flags & SHF_pagetable_dying);
-
+ if ( curr->domain == d )
+ curr->arch.paging.shadow.pagetable_dying
+ = !!(mfn_to_page(gmfn)->shadow_flags & SHF_pagetable_dying);
sp = mfn_to_page(smfn);
@@ -4290,6 +4292,8 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
int done = 0;
int flags;
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
+ struct domain *d = v->domain;
+ struct vcpu *curr = current;
mfn_t base_sl1mfn = sl1mfn; /* Because sl1mfn changes in the foreach */
#endif
@@ -4304,7 +4308,8 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
(void) shadow_set_l1e(v, sl1e, ro_sl1e, p2m_ram_rw, sl1mfn);
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
/* Remember the last shadow that we shot a writeable mapping in */
- v->arch.paging.shadow.last_writeable_pte_smfn = mfn_x(base_sl1mfn);
+ if ( curr->domain == d )
+ curr->arch.paging.shadow.last_writeable_pte_smfn = mfn_x(base_sl1mfn);
#endif
if ( (mfn_to_page(readonly_mfn)->u.inuse.type_info
& PGT_count_mask) == 0 )
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread
* [PATCH 05/20] x86/shadow: Alter shadow_hash_{lookup, insert, delete}() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (3 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 04/20] x86/shadow: Only apply shadow heuristics when in guest context Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 06/20] x86/shadow: Alter *_shadow_status() and make_fl1_shadow() " Andrew Cooper
` (16 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 11 ++++-------
xen/arch/x86/mm/shadow/multi.c | 22 +++++++++++++---------
xen/arch/x86/mm/shadow/private.h | 6 +++---
3 files changed, 20 insertions(+), 19 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 26dab30..80174df 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1878,11 +1878,10 @@ static void shadow_hash_teardown(struct domain *d)
}
-mfn_t shadow_hash_lookup(struct vcpu *v, unsigned long n, unsigned int t)
+mfn_t shadow_hash_lookup(struct domain *d, unsigned long n, unsigned int t)
/* Find an entry in the hash table. Returns the MFN of the shadow,
* or INVALID_MFN if it doesn't exist */
{
- struct domain *d = v->domain;
struct page_info *sp, *prev;
key_t key;
@@ -1932,11 +1931,10 @@ mfn_t shadow_hash_lookup(struct vcpu *v, unsigned long n, unsigned int t)
return _mfn(INVALID_MFN);
}
-void shadow_hash_insert(struct vcpu *v, unsigned long n, unsigned int t,
+void shadow_hash_insert(struct domain *d, unsigned long n, unsigned int t,
mfn_t smfn)
/* Put a mapping (n,t)->smfn into the hash table */
{
- struct domain *d = v->domain;
struct page_info *sp;
key_t key;
@@ -1958,11 +1956,10 @@ void shadow_hash_insert(struct vcpu *v, unsigned long n, unsigned int t,
sh_hash_audit_bucket(d, key);
}
-void shadow_hash_delete(struct vcpu *v, unsigned long n, unsigned int t,
+void shadow_hash_delete(struct domain *d, unsigned long n, unsigned int t,
mfn_t smfn)
/* Excise the mapping (n,t)->smfn from the hash table */
{
- struct domain *d = v->domain;
struct page_info *sp, *x;
key_t key;
@@ -2611,7 +2608,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
if( !(pg->count_info & PGC_page_table) \
|| !(pg->shadow_flags & (1 << t)) ) \
break; \
- smfn = shadow_hash_lookup(v, mfn_x(gmfn), t); \
+ smfn = shadow_hash_lookup(d, mfn_x(gmfn), t); \
if ( unlikely(!mfn_valid(smfn)) ) \
{ \
SHADOW_ERROR(": gmfn %#lx has flags %#"PRIx32 \
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index f532bff..1e6bc33 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -94,7 +94,8 @@ static inline mfn_t
get_fl1_shadow_status(struct vcpu *v, gfn_t gfn)
/* Look for FL1 shadows in the hash table */
{
- mfn_t smfn = shadow_hash_lookup(v, gfn_x(gfn), SH_type_fl1_shadow);
+ struct domain *d = v->domain;
+ mfn_t smfn = shadow_hash_lookup(d, gfn_x(gfn), SH_type_fl1_shadow);
ASSERT(!mfn_valid(smfn) || mfn_to_page(smfn)->u.sh.head);
return smfn;
}
@@ -103,7 +104,8 @@ static inline mfn_t
get_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
/* Look for shadows in the hash table */
{
- mfn_t smfn = shadow_hash_lookup(v, mfn_x(gmfn), shadow_type);
+ struct domain *d = v->domain;
+ mfn_t smfn = shadow_hash_lookup(d, mfn_x(gmfn), shadow_type);
ASSERT(!mfn_valid(smfn) || mfn_to_page(smfn)->u.sh.head);
perfc_incr(shadow_get_shadow_status);
return smfn;
@@ -113,11 +115,12 @@ static inline void
set_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
/* Put an FL1 shadow into the hash table */
{
+ struct domain *d = v->domain;
SHADOW_PRINTK("gfn=%"SH_PRI_gfn", type=%08x, smfn=%05lx\n",
gfn_x(gfn), SH_type_fl1_shadow, mfn_x(smfn));
ASSERT(mfn_to_page(smfn)->u.sh.head);
- shadow_hash_insert(v, gfn_x(gfn), SH_type_fl1_shadow, smfn);
+ shadow_hash_insert(d, gfn_x(gfn), SH_type_fl1_shadow, smfn);
}
static inline void
@@ -140,17 +143,18 @@ set_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
ASSERT(res == 1);
}
- shadow_hash_insert(v, mfn_x(gmfn), shadow_type, smfn);
+ shadow_hash_insert(d, mfn_x(gmfn), shadow_type, smfn);
}
static inline void
delete_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
/* Remove a shadow from the hash table */
{
+ struct domain *d = v->domain;
SHADOW_PRINTK("gfn=%"SH_PRI_gfn", type=%08x, smfn=%05lx\n",
gfn_x(gfn), SH_type_fl1_shadow, mfn_x(smfn));
ASSERT(mfn_to_page(smfn)->u.sh.head);
- shadow_hash_delete(v, gfn_x(gfn), SH_type_fl1_shadow, smfn);
+ shadow_hash_delete(d, gfn_x(gfn), SH_type_fl1_shadow, smfn);
}
static inline void
@@ -162,7 +166,7 @@ delete_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
d->domain_id, v->vcpu_id,
mfn_x(gmfn), shadow_type, mfn_x(smfn));
ASSERT(mfn_to_page(smfn)->u.sh.head);
- shadow_hash_delete(v, mfn_x(gmfn), shadow_type, smfn);
+ shadow_hash_delete(d, mfn_x(gmfn), shadow_type, smfn);
/* 32-on-64 PV guests don't own their l4 pages; see set_shadow_status */
if ( !is_pv_32on64_domain(d) || shadow_type != SH_type_l4_64_shadow )
put_page(mfn_to_page(gmfn));
@@ -4501,7 +4505,7 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
gl2a = guest_l3e_get_paddr(gl3e[i]);
gfn = gl2a >> PAGE_SHIFT;
gmfn = get_gfn_query_unlocked(d, gfn, &p2mt);
- smfn = shadow_hash_lookup(v, mfn_x(gmfn), SH_type_l2_pae_shadow);
+ smfn = shadow_hash_lookup(d, mfn_x(gmfn), SH_type_l2_pae_shadow);
}
if ( mfn_valid(smfn) )
@@ -4539,9 +4543,9 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
paging_lock(d);
#if GUEST_PAGING_LEVELS == 2
- smfn = shadow_hash_lookup(v, mfn_x(gmfn), SH_type_l2_32_shadow);
+ smfn = shadow_hash_lookup(d, mfn_x(gmfn), SH_type_l2_32_shadow);
#else
- smfn = shadow_hash_lookup(v, mfn_x(gmfn), SH_type_l4_64_shadow);
+ smfn = shadow_hash_lookup(d, mfn_x(gmfn), SH_type_l4_64_shadow);
#endif
if ( mfn_valid(smfn) )
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index baa271c..df1dd8c 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -343,10 +343,10 @@ shadow_size(unsigned int shadow_type)
*/
/* Hash table functions */
-mfn_t shadow_hash_lookup(struct vcpu *v, unsigned long n, unsigned int t);
-void shadow_hash_insert(struct vcpu *v,
+mfn_t shadow_hash_lookup(struct domain *d, unsigned long n, unsigned int t);
+void shadow_hash_insert(struct domain *d,
unsigned long n, unsigned int t, mfn_t smfn);
-void shadow_hash_delete(struct vcpu *v,
+void shadow_hash_delete(struct domain *d,
unsigned long n, unsigned int t, mfn_t smfn);
/* shadow promotion */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 06/20] x86/shadow: Alter *_shadow_status() and make_fl1_shadow() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (4 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 05/20] x86/shadow: Alter shadow_hash_{lookup, insert, delete}() to take a domain Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 07/20] x86/shadow: Alter sh_type_{is_pinnable, has_up_pointer}() " Andrew Cooper
` (15 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/multi.c | 99 +++++++++++++++++++---------------------
1 file changed, 48 insertions(+), 51 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 1e6bc33..154274f 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -91,20 +91,18 @@ static char *fetch_type_names[] = {
*/
static inline mfn_t
-get_fl1_shadow_status(struct vcpu *v, gfn_t gfn)
+get_fl1_shadow_status(struct domain *d, gfn_t gfn)
/* Look for FL1 shadows in the hash table */
{
- struct domain *d = v->domain;
mfn_t smfn = shadow_hash_lookup(d, gfn_x(gfn), SH_type_fl1_shadow);
ASSERT(!mfn_valid(smfn) || mfn_to_page(smfn)->u.sh.head);
return smfn;
}
static inline mfn_t
-get_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
+get_shadow_status(struct domain *d, mfn_t gmfn, u32 shadow_type)
/* Look for shadows in the hash table */
{
- struct domain *d = v->domain;
mfn_t smfn = shadow_hash_lookup(d, mfn_x(gmfn), shadow_type);
ASSERT(!mfn_valid(smfn) || mfn_to_page(smfn)->u.sh.head);
perfc_incr(shadow_get_shadow_status);
@@ -112,10 +110,9 @@ get_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
}
static inline void
-set_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
+set_fl1_shadow_status(struct domain *d, gfn_t gfn, mfn_t smfn)
/* Put an FL1 shadow into the hash table */
{
- struct domain *d = v->domain;
SHADOW_PRINTK("gfn=%"SH_PRI_gfn", type=%08x, smfn=%05lx\n",
gfn_x(gfn), SH_type_fl1_shadow, mfn_x(smfn));
@@ -124,15 +121,13 @@ set_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
}
static inline void
-set_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
+set_shadow_status(struct domain *d, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
/* Put a shadow into the hash table */
{
- struct domain *d = v->domain;
int res;
- SHADOW_PRINTK("d=%d, v=%d, gmfn=%05lx, type=%08x, smfn=%05lx\n",
- d->domain_id, v->vcpu_id, mfn_x(gmfn),
- shadow_type, mfn_x(smfn));
+ SHADOW_PRINTK("d=%d: gmfn=%lx, type=%08x, smfn=%lx\n",
+ d->domain_id, mfn_x(gmfn), shadow_type, mfn_x(smfn));
ASSERT(mfn_to_page(smfn)->u.sh.head);
@@ -147,10 +142,9 @@ set_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
}
static inline void
-delete_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
+delete_fl1_shadow_status(struct domain *d, gfn_t gfn, mfn_t smfn)
/* Remove a shadow from the hash table */
{
- struct domain *d = v->domain;
SHADOW_PRINTK("gfn=%"SH_PRI_gfn", type=%08x, smfn=%05lx\n",
gfn_x(gfn), SH_type_fl1_shadow, mfn_x(smfn));
ASSERT(mfn_to_page(smfn)->u.sh.head);
@@ -158,13 +152,11 @@ delete_fl1_shadow_status(struct vcpu *v, gfn_t gfn, mfn_t smfn)
}
static inline void
-delete_shadow_status(struct vcpu *v, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
+delete_shadow_status(struct domain *d, mfn_t gmfn, u32 shadow_type, mfn_t smfn)
/* Remove a shadow from the hash table */
{
- struct domain *d = v->domain;
- SHADOW_PRINTK("d=%d, v=%d, gmfn=%05lx, type=%08x, smfn=%05lx\n",
- d->domain_id, v->vcpu_id,
- mfn_x(gmfn), shadow_type, mfn_x(smfn));
+ SHADOW_PRINTK("d=%d: gmfn=%lx, type=%08x, smfn=%lx\n",
+ d->domain_id, mfn_x(gmfn), shadow_type, mfn_x(smfn));
ASSERT(mfn_to_page(smfn)->u.sh.head);
shadow_hash_delete(d, mfn_x(gmfn), shadow_type, smfn);
/* 32-on-64 PV guests don't own their l4 pages; see set_shadow_status */
@@ -330,6 +322,7 @@ gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
* through the audit mechanisms */
static void sh_audit_gw(struct vcpu *v, walk_t *gw)
{
+ struct domain *d = v->domain;
mfn_t smfn;
if ( !(SHADOW_AUDIT_ENABLE) )
@@ -337,33 +330,33 @@ static void sh_audit_gw(struct vcpu *v, walk_t *gw)
#if GUEST_PAGING_LEVELS >= 4 /* 64-bit only... */
if ( mfn_valid(gw->l4mfn)
- && mfn_valid((smfn = get_shadow_status(v, gw->l4mfn,
+ && mfn_valid((smfn = get_shadow_status(d, gw->l4mfn,
SH_type_l4_shadow))) )
(void) sh_audit_l4_table(v, smfn, _mfn(INVALID_MFN));
if ( mfn_valid(gw->l3mfn)
- && mfn_valid((smfn = get_shadow_status(v, gw->l3mfn,
+ && mfn_valid((smfn = get_shadow_status(d, gw->l3mfn,
SH_type_l3_shadow))) )
(void) sh_audit_l3_table(v, smfn, _mfn(INVALID_MFN));
#endif /* PAE or 64... */
if ( mfn_valid(gw->l2mfn) )
{
- if ( mfn_valid((smfn = get_shadow_status(v, gw->l2mfn,
+ if ( mfn_valid((smfn = get_shadow_status(d, gw->l2mfn,
SH_type_l2_shadow))) )
(void) sh_audit_l2_table(v, smfn, _mfn(INVALID_MFN));
#if GUEST_PAGING_LEVELS == 3
- if ( mfn_valid((smfn = get_shadow_status(v, gw->l2mfn,
+ if ( mfn_valid((smfn = get_shadow_status(d, gw->l2mfn,
SH_type_l2h_shadow))) )
(void) sh_audit_l2_table(v, smfn, _mfn(INVALID_MFN));
#endif
}
if ( mfn_valid(gw->l1mfn)
- && mfn_valid((smfn = get_shadow_status(v, gw->l1mfn,
+ && mfn_valid((smfn = get_shadow_status(d, gw->l1mfn,
SH_type_l1_shadow))) )
(void) sh_audit_l1_table(v, smfn, _mfn(INVALID_MFN));
else if ( (guest_l2e_get_flags(gw->l2e) & _PAGE_PRESENT)
&& (guest_l2e_get_flags(gw->l2e) & _PAGE_PSE)
&& mfn_valid(
- (smfn = get_fl1_shadow_status(v, guest_l2e_get_gfn(gw->l2e)))) )
+ (smfn = get_fl1_shadow_status(d, guest_l2e_get_gfn(gw->l2e)))) )
(void) sh_audit_fl1_table(v, smfn, _mfn(INVALID_MFN));
}
@@ -1566,22 +1559,21 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
}
shadow_promote(v, gmfn, shadow_type);
- set_shadow_status(v, gmfn, shadow_type, smfn);
+ set_shadow_status(d, gmfn, shadow_type, smfn);
return smfn;
}
/* Make a splintered superpage shadow */
static mfn_t
-make_fl1_shadow(struct vcpu *v, gfn_t gfn)
+make_fl1_shadow(struct domain *d, gfn_t gfn)
{
- struct domain *d = v->domain;
mfn_t smfn = shadow_alloc(d, SH_type_fl1_shadow, gfn_x(gfn));
SHADOW_DEBUG(MAKE_SHADOW, "(%" SH_PRI_gfn ")=>%" PRI_mfn "\n",
gfn_x(gfn), mfn_x(smfn));
- set_fl1_shadow_status(v, gfn, smfn);
+ set_fl1_shadow_status(d, gfn, smfn);
return smfn;
}
@@ -1677,6 +1669,7 @@ static shadow_l3e_t * shadow_get_and_create_l3e(struct vcpu *v,
fetch_type_t ft,
int *resync)
{
+ struct domain *d = v->domain;
mfn_t sl4mfn;
shadow_l4e_t *sl4e;
if ( !mfn_valid(gw->l3mfn) ) return NULL; /* No guest page. */
@@ -1693,7 +1686,7 @@ static shadow_l3e_t * shadow_get_and_create_l3e(struct vcpu *v,
int r;
shadow_l4e_t new_sl4e;
/* No l3 shadow installed: find and install it. */
- *sl3mfn = get_shadow_status(v, gw->l3mfn, SH_type_l3_shadow);
+ *sl3mfn = get_shadow_status(d, gw->l3mfn, SH_type_l3_shadow);
if ( !mfn_valid(*sl3mfn) )
{
/* No l3 shadow of this page exists at all: make one. */
@@ -1724,6 +1717,7 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
int *resync)
{
#if GUEST_PAGING_LEVELS >= 4 /* 64bit... */
+ struct domain *d = v->domain;
mfn_t sl3mfn = _mfn(INVALID_MFN);
shadow_l3e_t *sl3e;
if ( !mfn_valid(gw->l2mfn) ) return NULL; /* No guest page. */
@@ -1748,7 +1742,7 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
t = SH_type_l2h_shadow;
/* No l2 shadow installed: find and install it. */
- *sl2mfn = get_shadow_status(v, gw->l2mfn, t);
+ *sl2mfn = get_shadow_status(d, gw->l2mfn, t);
if ( !mfn_valid(*sl2mfn) )
{
/* No l2 shadow of this page exists at all: make one. */
@@ -1796,6 +1790,7 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
mfn_t *sl1mfn,
fetch_type_t ft)
{
+ struct domain *d = v->domain;
mfn_t sl2mfn;
int resync = 0;
shadow_l2e_t *sl2e;
@@ -1825,18 +1820,18 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
{
/* Splintering a superpage */
gfn_t l2gfn = guest_l2e_get_gfn(gw->l2e);
- *sl1mfn = get_fl1_shadow_status(v, l2gfn);
+ *sl1mfn = get_fl1_shadow_status(d, l2gfn);
if ( !mfn_valid(*sl1mfn) )
{
/* No fl1 shadow of this superpage exists at all: make one. */
- *sl1mfn = make_fl1_shadow(v, l2gfn);
+ *sl1mfn = make_fl1_shadow(d, l2gfn);
}
}
else
{
/* Shadowing an actual guest l1 table */
if ( !mfn_valid(gw->l1mfn) ) return NULL; /* No guest page. */
- *sl1mfn = get_shadow_status(v, gw->l1mfn, SH_type_l1_shadow);
+ *sl1mfn = get_shadow_status(d, gw->l1mfn, SH_type_l1_shadow);
if ( !mfn_valid(*sl1mfn) )
{
/* No l1 shadow of this page exists at all: make one. */
@@ -1897,7 +1892,7 @@ void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
/* Record that the guest page isn't shadowed any more (in this type) */
gmfn = backpointer(sp);
- delete_shadow_status(v, gmfn, t, smfn);
+ delete_shadow_status(d, gmfn, t, smfn);
shadow_demote(v, gmfn, t);
/* Decrement refcounts of all the old entries */
sl4mfn = smfn;
@@ -1929,7 +1924,7 @@ void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
/* Record that the guest page isn't shadowed any more (in this type) */
gmfn = backpointer(sp);
- delete_shadow_status(v, gmfn, t, smfn);
+ delete_shadow_status(d, gmfn, t, smfn);
shadow_demote(v, gmfn, t);
/* Decrement refcounts of all the old entries */
@@ -1967,7 +1962,7 @@ void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
/* Record that the guest page isn't shadowed any more (in this type) */
gmfn = backpointer(sp);
- delete_shadow_status(v, gmfn, t, smfn);
+ delete_shadow_status(d, gmfn, t, smfn);
shadow_demote(v, gmfn, t);
/* Decrement refcounts of all the old entries */
@@ -1999,12 +1994,12 @@ void sh_destroy_l1_shadow(struct vcpu *v, mfn_t smfn)
if ( t == SH_type_fl1_shadow )
{
gfn_t gfn = _gfn(sp->v.sh.back);
- delete_fl1_shadow_status(v, gfn, smfn);
+ delete_fl1_shadow_status(d, gfn, smfn);
}
else
{
mfn_t gmfn = backpointer(sp);
- delete_shadow_status(v, gmfn, t, smfn);
+ delete_shadow_status(d, gmfn, t, smfn);
shadow_demote(v, gmfn, t);
}
@@ -2148,7 +2143,7 @@ static int validate_gl4e(struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
gfn_t gl3gfn = guest_l4e_get_gfn(new_gl4e);
mfn_t gl3mfn = get_gfn_query_unlocked(d, gfn_x(gl3gfn), &p2mt);
if ( p2m_is_ram(p2mt) )
- sl3mfn = get_shadow_status(v, gl3mfn, SH_type_l3_shadow);
+ sl3mfn = get_shadow_status(d, gl3mfn, SH_type_l3_shadow);
else if ( p2mt != p2m_populate_on_demand )
result |= SHADOW_SET_ERROR;
@@ -2206,7 +2201,7 @@ static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
gfn_t gl2gfn = guest_l3e_get_gfn(new_gl3e);
mfn_t gl2mfn = get_gfn_query_unlocked(d, gfn_x(gl2gfn), &p2mt);
if ( p2m_is_ram(p2mt) )
- sl2mfn = get_shadow_status(v, gl2mfn, SH_type_l2_shadow);
+ sl2mfn = get_shadow_status(d, gl2mfn, SH_type_l2_shadow);
else if ( p2mt != p2m_populate_on_demand )
result |= SHADOW_SET_ERROR;
@@ -2242,21 +2237,21 @@ static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
{
// superpage -- need to look up the shadow L1 which holds the
// splitters...
- sl1mfn = get_fl1_shadow_status(v, gl1gfn);
+ sl1mfn = get_fl1_shadow_status(d, gl1gfn);
#if 0
// XXX - it's possible that we want to do some kind of prefetch
// for superpage fl1's here, but this is *not* on the demand path,
// so we'll hold off trying that for now...
//
if ( !mfn_valid(sl1mfn) )
- sl1mfn = make_fl1_shadow(v, gl1gfn);
+ sl1mfn = make_fl1_shadow(d, gl1gfn);
#endif
}
else
{
mfn_t gl1mfn = get_gfn_query_unlocked(d, gfn_x(gl1gfn), &p2mt);
if ( p2m_is_ram(p2mt) )
- sl1mfn = get_shadow_status(v, gl1mfn, SH_type_l1_shadow);
+ sl1mfn = get_shadow_status(d, gl1mfn, SH_type_l1_shadow);
else if ( p2mt != p2m_populate_on_demand )
result |= SHADOW_SET_ERROR;
}
@@ -2327,7 +2322,7 @@ void sh_resync_l1(struct vcpu *v, mfn_t gl1mfn, mfn_t snpmfn)
ASSERT(mfn_valid(snpmfn));
- sl1mfn = get_shadow_status(v, gl1mfn, SH_type_l1_shadow);
+ sl1mfn = get_shadow_status(d, gl1mfn, SH_type_l1_shadow);
ASSERT(mfn_valid(sl1mfn)); /* Otherwise we would not have been called */
snp = sh_map_domain_page(snpmfn);
@@ -2368,13 +2363,14 @@ void sh_resync_l1(struct vcpu *v, mfn_t gl1mfn, mfn_t snpmfn)
* called in the *mode* of the vcpu that unsynced it. Clear? Good. */
int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
{
+ struct domain *d = v->domain;
struct page_info *sp;
mfn_t smfn;
if ( !sh_type_has_up_pointer(v, SH_type_l1_shadow) )
return 0;
- smfn = get_shadow_status(v, gl1mfn, SH_type_l1_shadow);
+ smfn = get_shadow_status(d, gl1mfn, SH_type_l1_shadow);
ASSERT(mfn_valid(smfn)); /* Otherwise we would not have been called */
/* Up to l2 */
@@ -2431,6 +2427,7 @@ sh_map_and_validate(struct vcpu *v, mfn_t gmfn,
mfn_t smfn, void *se))
/* Generic function for mapping and validating. */
{
+ struct domain *d = v->domain;
mfn_t smfn, smfn2, map_mfn;
shadow_l1e_t *sl1p;
u32 shadow_idx, guest_idx;
@@ -2443,7 +2440,7 @@ sh_map_and_validate(struct vcpu *v, mfn_t gmfn,
ASSERT(size + (((unsigned long)new_gp) & ~PAGE_MASK) <= PAGE_SIZE);
/* Map the shadow page */
- smfn = get_shadow_status(v, gmfn, sh_type);
+ smfn = get_shadow_status(d, gmfn, sh_type);
ASSERT(mfn_valid(smfn)); /* Otherwise we would not have been called */
guest_idx = guest_index(new_gp);
map_mfn = smfn;
@@ -3857,7 +3854,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
}
/* Guest mfn is valid: shadow it and install the shadow */
- smfn = get_shadow_status(v, gmfn, root_type);
+ smfn = get_shadow_status(d, gmfn, root_type);
if ( !mfn_valid(smfn) )
{
/* Make sure there's enough free shadow memory. */
@@ -5048,8 +5045,8 @@ int sh_audit_l2_table(struct vcpu *v, mfn_t sl2mfn, mfn_t x)
gfn = guest_l2e_get_gfn(*gl2e);
mfn = shadow_l2e_get_mfn(*sl2e);
gmfn = (guest_l2e_get_flags(*gl2e) & _PAGE_PSE)
- ? get_fl1_shadow_status(v, gfn)
- : get_shadow_status(v,
+ ? get_fl1_shadow_status(d, gfn)
+ : get_shadow_status(d,
get_gfn_query_unlocked(d, gfn_x(gfn),
&p2mt), SH_type_l1_shadow);
if ( mfn_x(gmfn) != mfn_x(mfn) )
@@ -5099,7 +5096,7 @@ int sh_audit_l3_table(struct vcpu *v, mfn_t sl3mfn, mfn_t x)
{
gfn = guest_l3e_get_gfn(*gl3e);
mfn = shadow_l3e_get_mfn(*sl3e);
- gmfn = get_shadow_status(v, get_gfn_query_unlocked(
+ gmfn = get_shadow_status(d, get_gfn_query_unlocked(
d, gfn_x(gfn), &p2mt),
((GUEST_PAGING_LEVELS == 3 ||
is_pv_32on64_vcpu(v))
@@ -5149,7 +5146,7 @@ int sh_audit_l4_table(struct vcpu *v, mfn_t sl4mfn, mfn_t x)
{
gfn = guest_l4e_get_gfn(*gl4e);
mfn = shadow_l4e_get_mfn(*sl4e);
- gmfn = get_shadow_status(v, get_gfn_query_unlocked(
+ gmfn = get_shadow_status(d, get_gfn_query_unlocked(
d, gfn_x(gfn), &p2mt),
SH_type_l3_shadow);
if ( mfn_x(gmfn) != mfn_x(mfn) )
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 07/20] x86/shadow: Alter sh_type_{is_pinnable, has_up_pointer}() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (5 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 06/20] x86/shadow: Alter *_shadow_status() and make_fl1_shadow() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 08/20] x86/shadow: Alter OOS functions " Andrew Cooper
` (14 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 7 ++++---
xen/arch/x86/mm/shadow/multi.c | 10 +++++-----
xen/arch/x86/mm/shadow/private.h | 18 ++++++++++--------
3 files changed, 19 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 80174df..bdb19fb 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -2472,6 +2472,7 @@ static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
/* Follow this shadow's up-pointer, if it has one, and remove the reference
* found there. Returns 1 if that was the only reference to this shadow */
{
+ struct domain *d = v->domain;
struct page_info *sp = mfn_to_page(smfn);
mfn_t pmfn;
void *vaddr;
@@ -2479,7 +2480,7 @@ static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
ASSERT(sp->u.sh.type > 0);
ASSERT(sp->u.sh.type < SH_type_max_shadow);
- ASSERT(sh_type_has_up_pointer(v, sp->u.sh.type));
+ ASSERT(sh_type_has_up_pointer(d, sp->u.sh.type));
if (sp->up == 0) return 0;
pmfn = _mfn(sp->up >> PAGE_SHIFT);
@@ -2616,9 +2617,9 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
mfn_x(gmfn), (uint32_t)pg->shadow_flags, t); \
break; \
} \
- if ( sh_type_is_pinnable(v, t) ) \
+ if ( sh_type_is_pinnable(d, t) ) \
sh_unpin(v, smfn); \
- else if ( sh_type_has_up_pointer(v, t) ) \
+ else if ( sh_type_has_up_pointer(d, t) ) \
sh_remove_shadow_via_pointer(v, smfn); \
if( !fast \
&& (pg->count_info & PGC_page_table) \
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 154274f..ea3b520 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -903,7 +903,7 @@ static int shadow_set_l4e(struct vcpu *v,
mfn_t sl3mfn = shadow_l4e_get_mfn(new_sl4e);
ok = sh_get_ref(v, sl3mfn, paddr);
/* Are we pinning l3 shadows to handle wierd linux behaviour? */
- if ( sh_type_is_pinnable(v, SH_type_l3_64_shadow) )
+ if ( sh_type_is_pinnable(d, SH_type_l3_64_shadow) )
ok |= sh_pin(v, sl3mfn);
if ( !ok )
{
@@ -1501,7 +1501,7 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
SHADOW_DEBUG(MAKE_SHADOW, "(%05lx, %u)=>%05lx\n",
mfn_x(gmfn), shadow_type, mfn_x(smfn));
- if ( sh_type_has_up_pointer(v, shadow_type) )
+ if ( sh_type_has_up_pointer(d, shadow_type) )
/* Lower-level shadow, not yet linked form a higher level */
mfn_to_page(smfn)->up = 0;
@@ -2367,7 +2367,7 @@ int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
struct page_info *sp;
mfn_t smfn;
- if ( !sh_type_has_up_pointer(v, SH_type_l1_shadow) )
+ if ( !sh_type_has_up_pointer(d, SH_type_l1_shadow) )
return 0;
smfn = get_shadow_status(d, gl1mfn, SH_type_l1_shadow);
@@ -2383,7 +2383,7 @@ int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
#if (SHADOW_PAGING_LEVELS == 4)
/* up to l3 */
sp = mfn_to_page(smfn);
- ASSERT(sh_type_has_up_pointer(v, SH_type_l2_shadow));
+ ASSERT(sh_type_has_up_pointer(d, SH_type_l2_shadow));
if ( sp->u.sh.count != 1 || !sp->up )
return 0;
smfn = _mfn(sp->up >> PAGE_SHIFT);
@@ -2392,7 +2392,7 @@ int sh_safe_not_to_sync(struct vcpu *v, mfn_t gl1mfn)
/* up to l4 */
sp = mfn_to_page(smfn);
if ( sp->u.sh.count != 1
- || !sh_type_has_up_pointer(v, SH_type_l3_64_shadow) || !sp->up )
+ || !sh_type_has_up_pointer(d, SH_type_l3_64_shadow) || !sp->up )
return 0;
smfn = _mfn(sp->up >> PAGE_SHIFT);
ASSERT(mfn_valid(smfn));
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index df1dd8c..8c06775 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -195,7 +195,7 @@ extern void shadow_audit_tables(struct vcpu *v);
* What counts as a pinnable shadow?
*/
-static inline int sh_type_is_pinnable(struct vcpu *v, unsigned int t)
+static inline int sh_type_is_pinnable(struct domain *d, unsigned int t)
{
/* Top-level shadow types in each mode can be pinned, so that they
* persist even when not currently in use in a guest CR3 */
@@ -211,7 +211,7 @@ static inline int sh_type_is_pinnable(struct vcpu *v, unsigned int t)
* page. When we're shadowing those kernels, we have to pin l3
* shadows so they don't just evaporate on every context switch.
* For all other guests, we'd rather use the up-pointer field in l3s. */
- if ( unlikely((v->domain->arch.paging.shadow.opt_flags & SHOPT_LINUX_L3_TOPLEVEL)
+ if ( unlikely((d->arch.paging.shadow.opt_flags & SHOPT_LINUX_L3_TOPLEVEL)
&& t == SH_type_l3_64_shadow) )
return 1;
#endif
@@ -220,7 +220,7 @@ static inline int sh_type_is_pinnable(struct vcpu *v, unsigned int t)
return 0;
}
-static inline int sh_type_has_up_pointer(struct vcpu *v, unsigned int t)
+static inline int sh_type_has_up_pointer(struct domain *d, unsigned int t)
{
/* Multi-page shadows don't have up-pointers */
if ( t == SH_type_l1_32_shadow
@@ -228,7 +228,7 @@ static inline int sh_type_has_up_pointer(struct vcpu *v, unsigned int t)
|| t == SH_type_l2_32_shadow )
return 0;
/* Pinnable shadows don't have up-pointers either */
- return !sh_type_is_pinnable(v, t);
+ return !sh_type_is_pinnable(d, t);
}
static inline void sh_terminate_list(struct page_list_head *tmp_list)
@@ -540,6 +540,7 @@ void sh_destroy_shadow(struct vcpu *v, mfn_t smfn);
* Returns 0 for failure, 1 for success. */
static inline int sh_get_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
{
+ struct domain *d = v->domain;
u32 x, nx;
struct page_info *sp = mfn_to_page(smfn);
@@ -561,7 +562,7 @@ static inline int sh_get_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
/* We remember the first shadow entry that points to each shadow. */
if ( entry_pa != 0
- && sh_type_has_up_pointer(v, sp->u.sh.type)
+ && sh_type_has_up_pointer(d, sp->u.sh.type)
&& sp->up == 0 )
sp->up = entry_pa;
@@ -573,6 +574,7 @@ static inline int sh_get_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
* physical address of the shadow entry that held this reference. */
static inline void sh_put_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
{
+ struct domain *d = v->domain;
u32 x, nx;
struct page_info *sp = mfn_to_page(smfn);
@@ -582,7 +584,7 @@ static inline void sh_put_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
/* If this is the entry in the up-pointer, remove it */
if ( entry_pa != 0
- && sh_type_has_up_pointer(v, sp->u.sh.type)
+ && sh_type_has_up_pointer(d, sp->u.sh.type)
&& sp->up == entry_pa )
sp->up = 0;
@@ -656,7 +658,7 @@ static inline int sh_pin(struct vcpu *v, mfn_t smfn)
sp[0] = mfn_to_page(smfn);
pages = shadow_size(sp[0]->u.sh.type);
already_pinned = sp[0]->u.sh.pinned;
- ASSERT(sh_type_is_pinnable(v, sp[0]->u.sh.type));
+ ASSERT(sh_type_is_pinnable(d, sp[0]->u.sh.type));
ASSERT(sp[0]->u.sh.head);
pin_list = &d->arch.paging.shadow.pinned_shadows;
@@ -704,7 +706,7 @@ static inline void sh_unpin(struct vcpu *v, mfn_t smfn)
ASSERT(mfn_valid(smfn));
sp = mfn_to_page(smfn);
head_type = sp->u.sh.type;
- ASSERT(sh_type_is_pinnable(v, sp->u.sh.type));
+ ASSERT(sh_type_is_pinnable(d, sp->u.sh.type));
ASSERT(sp->u.sh.head);
if ( !sp->u.sh.pinned )
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 08/20] x86/shadow: Alter OOS functions to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (6 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 07/20] x86/shadow: Alter sh_type_{is_pinnable, has_up_pointer}() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 09/20] x86/shadow: Alter shadow_{pro, de}mote() " Andrew Cooper
` (13 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 23 ++++++++++++-----------
xen/arch/x86/mm/shadow/multi.c | 19 ++++++++++++-------
xen/arch/x86/mm/shadow/private.h | 6 +++---
3 files changed, 27 insertions(+), 21 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index bdb19fb..6945dfe 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -603,13 +603,13 @@ static inline int oos_fixup_flush_gmfn(struct vcpu *v, mfn_t gmfn,
return 1;
}
-void oos_fixup_add(struct vcpu *v, mfn_t gmfn,
+void oos_fixup_add(struct domain *d, mfn_t gmfn,
mfn_t smfn, unsigned long off)
{
int idx, next;
mfn_t *oos;
struct oos_fixup *oos_fixup;
- struct domain *d = v->domain;
+ struct vcpu *v;
perfc_incr(shadow_oos_fixup_add);
@@ -788,13 +788,13 @@ static void oos_hash_add(struct vcpu *v, mfn_t gmfn)
}
/* Remove an MFN from the list of out-of-sync guest pagetables */
-static void oos_hash_remove(struct vcpu *v, mfn_t gmfn)
+static void oos_hash_remove(struct domain *d, mfn_t gmfn)
{
int idx;
mfn_t *oos;
- struct domain *d = v->domain;
+ struct vcpu *v;
- SHADOW_PRINTK("%pv gmfn %lx\n", v, mfn_x(gmfn));
+ SHADOW_PRINTK("d%d gmfn %lx\n", d->domain_id, mfn_x(gmfn));
for_each_vcpu(d, v)
{
@@ -813,12 +813,12 @@ static void oos_hash_remove(struct vcpu *v, mfn_t gmfn)
BUG();
}
-mfn_t oos_snapshot_lookup(struct vcpu *v, mfn_t gmfn)
+mfn_t oos_snapshot_lookup(struct domain *d, mfn_t gmfn)
{
int idx;
mfn_t *oos;
mfn_t *oos_snapshot;
- struct domain *d = v->domain;
+ struct vcpu *v;
for_each_vcpu(d, v)
{
@@ -839,13 +839,13 @@ mfn_t oos_snapshot_lookup(struct vcpu *v, mfn_t gmfn)
}
/* Pull a single guest page back into sync */
-void sh_resync(struct vcpu *v, mfn_t gmfn)
+void sh_resync(struct domain *d, mfn_t gmfn)
{
int idx;
mfn_t *oos;
mfn_t *oos_snapshot;
struct oos_fixup *oos_fixup;
- struct domain *d = v->domain;
+ struct vcpu *v;
for_each_vcpu(d, v)
{
@@ -1000,7 +1000,7 @@ void shadow_promote(struct vcpu *v, mfn_t gmfn, unsigned int type)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Is the page already shadowed and out of sync? */
if ( page_is_out_of_sync(page) )
- sh_resync(v, gmfn);
+ sh_resync(d, gmfn);
#endif
/* We should never try to promote a gmfn that has writeable mappings */
@@ -1019,6 +1019,7 @@ void shadow_promote(struct vcpu *v, mfn_t gmfn, unsigned int type)
void shadow_demote(struct vcpu *v, mfn_t gmfn, u32 type)
{
+ struct domain *d = v->domain;
struct page_info *page = mfn_to_page(gmfn);
ASSERT(test_bit(_PGC_page_table, &page->count_info));
@@ -1032,7 +1033,7 @@ void shadow_demote(struct vcpu *v, mfn_t gmfn, u32 type)
/* Was the page out of sync? */
if ( page_is_out_of_sync(page) )
{
- oos_hash_remove(v, gmfn);
+ oos_hash_remove(d, gmfn);
}
#endif
clear_bit(_PGC_page_table, &page->count_info);
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index ea3b520..82759a6 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -278,6 +278,11 @@ shadow_check_gl1e(struct vcpu *v, walk_t *gw)
static inline uint32_t
gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
{
+#if GUEST_PAGING_LEVELS >= 3 /* PAE or 64... */
+#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
+ struct domain *d = v->domain;
+#endif
+#endif
uint32_t rc = 0;
#if GUEST_PAGING_LEVELS >= 3 /* PAE or 64... */
@@ -285,7 +290,7 @@ gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
if ( mfn_is_out_of_sync(gw->l3mfn) )
{
- sh_resync(v, gw->l3mfn);
+ sh_resync(d, gw->l3mfn);
rc = GW_RMWR_REWALK;
}
else
@@ -297,7 +302,7 @@ gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
if ( mfn_is_out_of_sync(gw->l2mfn) )
{
- sh_resync(v, gw->l2mfn);
+ sh_resync(d, gw->l2mfn);
rc |= GW_RMWR_REWALK;
}
else
@@ -1030,7 +1035,7 @@ static int shadow_set_l2e(struct vcpu *v,
OOS. */
if ( (sp->u.sh.type != SH_type_fl1_shadow) && mfn_valid(gl1mfn)
&& mfn_is_out_of_sync(gl1mfn) )
- sh_resync(v, gl1mfn);
+ sh_resync(d, gl1mfn);
}
#endif
#if GUEST_PAGING_LEVELS == 2
@@ -1178,7 +1183,7 @@ static int shadow_set_l1e(struct vcpu *v,
if ( mfn_valid(new_gmfn) && mfn_oos_may_write(new_gmfn)
&& ((shadow_l1e_get_flags(new_sl1e) & (_PAGE_RW|_PAGE_PRESENT))
== (_PAGE_RW|_PAGE_PRESENT)) )
- oos_fixup_add(v, new_gmfn, sl1mfn, pgentry_ptr_to_slot(sl1e));
+ oos_fixup_add(d, new_gmfn, sl1mfn, pgentry_ptr_to_slot(sl1e));
#endif
old_sl1e = *sl1e;
@@ -2291,7 +2296,7 @@ static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
&& mfn_is_out_of_sync(gl1mfn) )
{
/* Update the OOS snapshot. */
- mfn_t snpmfn = oos_snapshot_lookup(v, gl1mfn);
+ mfn_t snpmfn = oos_snapshot_lookup(d, gl1mfn);
guest_l1e_t *snp;
ASSERT(mfn_valid(snpmfn));
@@ -2629,7 +2634,7 @@ static void sh_prefetch(struct vcpu *v, walk_t *gw,
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
if ( mfn_is_out_of_sync(gw->l1mfn) )
{
- mfn_t snpmfn = oos_snapshot_lookup(v, gw->l1mfn);
+ mfn_t snpmfn = oos_snapshot_lookup(d, gw->l1mfn);
ASSERT(mfn_valid(snpmfn));
snpl1p = sh_map_domain_page(snpmfn);
@@ -3167,7 +3172,7 @@ static int sh_page_fault(struct vcpu *v,
&& mfn_is_out_of_sync(gw.l1mfn) )
{
/* Update the OOS snapshot. */
- mfn_t snpmfn = oos_snapshot_lookup(v, gw.l1mfn);
+ mfn_t snpmfn = oos_snapshot_lookup(d, gw.l1mfn);
guest_l1e_t *snp;
ASSERT(mfn_valid(snpmfn));
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index 8c06775..7abb0e0 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -401,9 +401,9 @@ void shadow_unhook_mappings(struct vcpu *v, mfn_t smfn, int user_only);
int sh_unsync(struct vcpu *v, mfn_t gmfn);
/* Pull an out-of-sync page back into sync. */
-void sh_resync(struct vcpu *v, mfn_t gmfn);
+void sh_resync(struct domain *d, mfn_t gmfn);
-void oos_fixup_add(struct vcpu *v, mfn_t gmfn, mfn_t smfn, unsigned long off);
+void oos_fixup_add(struct domain *d, mfn_t gmfn, mfn_t smfn, unsigned long off);
int sh_remove_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
mfn_t smfn, unsigned long offset);
@@ -432,7 +432,7 @@ shadow_sync_other_vcpus(struct vcpu *v)
}
void oos_audit_hash_is_present(struct domain *d, mfn_t gmfn);
-mfn_t oos_snapshot_lookup(struct vcpu *v, mfn_t gmfn);
+mfn_t oos_snapshot_lookup(struct domain *d, mfn_t gmfn);
#endif /* (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC) */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 09/20] x86/shadow: Alter shadow_{pro, de}mote() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (7 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 08/20] x86/shadow: Alter OOS functions " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 10/20] x86/shadow: Alter sh_put_ref() and shadow destroy functions " Andrew Cooper
` (12 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 6 ++----
xen/arch/x86/mm/shadow/multi.c | 10 +++++-----
xen/arch/x86/mm/shadow/private.h | 4 ++--
3 files changed, 9 insertions(+), 11 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 6945dfe..c6b8e6f 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -990,9 +990,8 @@ int sh_unsync(struct vcpu *v, mfn_t gmfn)
* involves making sure there are no writable mappings available to the guest
* for this page.
*/
-void shadow_promote(struct vcpu *v, mfn_t gmfn, unsigned int type)
+void shadow_promote(struct domain *d, mfn_t gmfn, unsigned int type)
{
- struct domain *d = v->domain;
struct page_info *page = mfn_to_page(gmfn);
ASSERT(mfn_valid(gmfn));
@@ -1017,9 +1016,8 @@ void shadow_promote(struct vcpu *v, mfn_t gmfn, unsigned int type)
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_PROMOTE);
}
-void shadow_demote(struct vcpu *v, mfn_t gmfn, u32 type)
+void shadow_demote(struct domain *d, mfn_t gmfn, u32 type)
{
- struct domain *d = v->domain;
struct page_info *page = mfn_to_page(gmfn);
ASSERT(test_bit(_PGC_page_table, &page->count_info));
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 82759a6..f2dea16 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -1563,7 +1563,7 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
}
}
- shadow_promote(v, gmfn, shadow_type);
+ shadow_promote(d, gmfn, shadow_type);
set_shadow_status(d, gmfn, shadow_type, smfn);
return smfn;
@@ -1898,7 +1898,7 @@ void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
/* Record that the guest page isn't shadowed any more (in this type) */
gmfn = backpointer(sp);
delete_shadow_status(d, gmfn, t, smfn);
- shadow_demote(v, gmfn, t);
+ shadow_demote(d, gmfn, t);
/* Decrement refcounts of all the old entries */
sl4mfn = smfn;
SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, d, {
@@ -1930,7 +1930,7 @@ void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
/* Record that the guest page isn't shadowed any more (in this type) */
gmfn = backpointer(sp);
delete_shadow_status(d, gmfn, t, smfn);
- shadow_demote(v, gmfn, t);
+ shadow_demote(d, gmfn, t);
/* Decrement refcounts of all the old entries */
sl3mfn = smfn;
@@ -1968,7 +1968,7 @@ void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
/* Record that the guest page isn't shadowed any more (in this type) */
gmfn = backpointer(sp);
delete_shadow_status(d, gmfn, t, smfn);
- shadow_demote(v, gmfn, t);
+ shadow_demote(d, gmfn, t);
/* Decrement refcounts of all the old entries */
sl2mfn = smfn;
@@ -2005,7 +2005,7 @@ void sh_destroy_l1_shadow(struct vcpu *v, mfn_t smfn)
{
mfn_t gmfn = backpointer(sp);
delete_shadow_status(d, gmfn, t, smfn);
- shadow_demote(v, gmfn, t);
+ shadow_demote(d, gmfn, t);
}
if ( shadow_mode_refcounts(d) )
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index 7abb0e0..3820d9e 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -350,8 +350,8 @@ void shadow_hash_delete(struct domain *d,
unsigned long n, unsigned int t, mfn_t smfn);
/* shadow promotion */
-void shadow_promote(struct vcpu *v, mfn_t gmfn, u32 type);
-void shadow_demote(struct vcpu *v, mfn_t gmfn, u32 type);
+void shadow_promote(struct domain *d, mfn_t gmfn, u32 type);
+void shadow_demote(struct domain *d, mfn_t gmfn, u32 type);
/* Shadow page allocation functions */
void shadow_prealloc(struct domain *d, u32 shadow_type, unsigned int count);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 10/20] x86/shadow: Alter sh_put_ref() and shadow destroy functions to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (8 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 09/20] x86/shadow: Alter shadow_{pro, de}mote() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 11/20] x86/shadow: Alter sh_get_ref() and sh_{, un}pin() " Andrew Cooper
` (11 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 19 +++++++++----------
xen/arch/x86/mm/shadow/multi.c | 30 +++++++++++++-----------------
xen/arch/x86/mm/shadow/multi.h | 8 ++++----
xen/arch/x86/mm/shadow/private.h | 9 ++++-----
4 files changed, 30 insertions(+), 36 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index c6b8e6f..e2ea6cb 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -2052,9 +2052,8 @@ static void hash_vcpu_foreach(struct vcpu *v, unsigned int callback_mask,
* which will decrement refcounts appropriately and return memory to the
* free pool. */
-void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
+void sh_destroy_shadow(struct domain *d, mfn_t smfn)
{
- struct domain *d = v->domain;
struct page_info *sp = mfn_to_page(smfn);
unsigned int t = sp->u.sh.type;
@@ -2076,36 +2075,36 @@ void sh_destroy_shadow(struct vcpu *v, mfn_t smfn)
{
case SH_type_l1_32_shadow:
case SH_type_fl1_32_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, 2)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, 2)(d, smfn);
break;
case SH_type_l2_32_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, 2)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, 2)(d, smfn);
break;
case SH_type_l1_pae_shadow:
case SH_type_fl1_pae_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, 3)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, 3)(d, smfn);
break;
case SH_type_l2_pae_shadow:
case SH_type_l2h_pae_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, 3)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, 3)(d, smfn);
break;
case SH_type_l1_64_shadow:
case SH_type_fl1_64_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, 4)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, 4)(d, smfn);
break;
case SH_type_l2h_64_shadow:
ASSERT(is_pv_32on64_domain(d));
/* Fall through... */
case SH_type_l2_64_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, 4)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, 4)(d, smfn);
break;
case SH_type_l3_64_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l3_shadow, 4)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l3_shadow, 4)(d, smfn);
break;
case SH_type_l4_64_shadow:
- SHADOW_INTERNAL_NAME(sh_destroy_l4_shadow, 4)(v, smfn);
+ SHADOW_INTERNAL_NAME(sh_destroy_l4_shadow, 4)(d, smfn);
break;
default:
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index f2dea16..7d82d90 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -931,7 +931,7 @@ static int shadow_set_l4e(struct vcpu *v,
{
flags |= SHADOW_SET_FLUSH;
}
- sh_put_ref(v, osl3mfn, paddr);
+ sh_put_ref(d, osl3mfn, paddr);
}
return flags;
}
@@ -977,7 +977,7 @@ static int shadow_set_l3e(struct vcpu *v,
{
flags |= SHADOW_SET_FLUSH;
}
- sh_put_ref(v, osl2mfn, paddr);
+ sh_put_ref(d, osl2mfn, paddr);
}
return flags;
}
@@ -1063,7 +1063,7 @@ static int shadow_set_l2e(struct vcpu *v,
{
flags |= SHADOW_SET_FLUSH;
}
- sh_put_ref(v, osl1mfn, paddr);
+ sh_put_ref(d, osl1mfn, paddr);
}
return flags;
}
@@ -1882,9 +1882,8 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
*/
#if GUEST_PAGING_LEVELS >= 4
-void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
+void sh_destroy_l4_shadow(struct domain *d, mfn_t smfn)
{
- struct domain *d = v->domain;
shadow_l4e_t *sl4e;
struct page_info *sp = mfn_to_page(smfn);
u32 t = sp->u.sh.type;
@@ -1904,7 +1903,7 @@ void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, d, {
if ( shadow_l4e_get_flags(*sl4e) & _PAGE_PRESENT )
{
- sh_put_ref(v, shadow_l4e_get_mfn(*sl4e),
+ sh_put_ref(d, shadow_l4e_get_mfn(*sl4e),
(((paddr_t)mfn_x(sl4mfn)) << PAGE_SHIFT)
| ((unsigned long)sl4e & ~PAGE_MASK));
}
@@ -1914,9 +1913,8 @@ void sh_destroy_l4_shadow(struct vcpu *v, mfn_t smfn)
shadow_free(d, smfn);
}
-void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
+void sh_destroy_l3_shadow(struct domain *d, mfn_t smfn)
{
- struct domain *d = v->domain;
shadow_l3e_t *sl3e;
struct page_info *sp = mfn_to_page(smfn);
u32 t = sp->u.sh.type;
@@ -1936,7 +1934,7 @@ void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
sl3mfn = smfn;
SHADOW_FOREACH_L3E(sl3mfn, sl3e, 0, 0, {
if ( shadow_l3e_get_flags(*sl3e) & _PAGE_PRESENT )
- sh_put_ref(v, shadow_l3e_get_mfn(*sl3e),
+ sh_put_ref(d, shadow_l3e_get_mfn(*sl3e),
(((paddr_t)mfn_x(sl3mfn)) << PAGE_SHIFT)
| ((unsigned long)sl3e & ~PAGE_MASK));
});
@@ -1947,9 +1945,8 @@ void sh_destroy_l3_shadow(struct vcpu *v, mfn_t smfn)
#endif /* GUEST_PAGING_LEVELS >= 4 */
-void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
+void sh_destroy_l2_shadow(struct domain *d, mfn_t smfn)
{
- struct domain *d = v->domain;
shadow_l2e_t *sl2e;
struct page_info *sp = mfn_to_page(smfn);
u32 t = sp->u.sh.type;
@@ -1974,7 +1971,7 @@ void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
sl2mfn = smfn;
SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( shadow_l2e_get_flags(*sl2e) & _PAGE_PRESENT )
- sh_put_ref(v, shadow_l2e_get_mfn(*sl2e),
+ sh_put_ref(d, shadow_l2e_get_mfn(*sl2e),
(((paddr_t)mfn_x(sl2mfn)) << PAGE_SHIFT)
| ((unsigned long)sl2e & ~PAGE_MASK));
});
@@ -1983,9 +1980,8 @@ void sh_destroy_l2_shadow(struct vcpu *v, mfn_t smfn)
shadow_free(d, smfn);
}
-void sh_destroy_l1_shadow(struct vcpu *v, mfn_t smfn)
+void sh_destroy_l1_shadow(struct domain *d, mfn_t smfn)
{
- struct domain *d = v->domain;
shadow_l1e_t *sl1e;
struct page_info *sp = mfn_to_page(smfn);
u32 t = sp->u.sh.type;
@@ -3799,6 +3795,7 @@ sh_update_linear_entries(struct vcpu *v)
static void
sh_detach_old_tables(struct vcpu *v)
{
+ struct domain *d = v->domain;
mfn_t smfn;
int i = 0;
@@ -3812,7 +3809,6 @@ sh_detach_old_tables(struct vcpu *v)
#else
if ( v->arch.paging.shadow.guest_vtable )
{
- struct domain *d = v->domain;
if ( shadow_mode_external(d) || shadow_mode_translate(d) )
sh_unmap_domain_page_global(v->arch.paging.shadow.guest_vtable);
v->arch.paging.shadow.guest_vtable = NULL;
@@ -3831,7 +3827,7 @@ sh_detach_old_tables(struct vcpu *v)
{
smfn = pagetable_get_mfn(v->arch.shadow_table[i]);
if ( mfn_x(smfn) )
- sh_put_ref(v, smfn, 0);
+ sh_put_ref(d, smfn, 0);
v->arch.shadow_table[i] = pagetable_null();
}
}
@@ -3904,7 +3900,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
SHADOW_ERROR("can't re-pin %#lx\n", mfn_x(old_smfn));
domain_crash(d);
}
- sh_put_ref(v, old_smfn, 0);
+ sh_put_ref(d, old_smfn, 0);
}
}
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 7f829fd..614103d 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -39,16 +39,16 @@ SHADOW_INTERNAL_NAME(sh_map_and_validate_gl4e, GUEST_LEVELS)(
extern void
SHADOW_INTERNAL_NAME(sh_destroy_l1_shadow, GUEST_LEVELS)(
- struct vcpu *v, mfn_t smfn);
+ struct domain *d, mfn_t smfn);
extern void
SHADOW_INTERNAL_NAME(sh_destroy_l2_shadow, GUEST_LEVELS)(
- struct vcpu *v, mfn_t smfn);
+ struct domain *d, mfn_t smfn);
extern void
SHADOW_INTERNAL_NAME(sh_destroy_l3_shadow, GUEST_LEVELS)(
- struct vcpu *v, mfn_t smfn);
+ struct domain *d, mfn_t smfn);
extern void
SHADOW_INTERNAL_NAME(sh_destroy_l4_shadow, GUEST_LEVELS)(
- struct vcpu *v, mfn_t smfn);
+ struct domain *d, mfn_t smfn);
extern void
SHADOW_INTERNAL_NAME(sh_unhook_32b_mappings, GUEST_LEVELS)
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index 3820d9e..a848c94 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -532,7 +532,7 @@ sh_unmap_domain_page_global(void *p)
/**************************************************************************/
/* Shadow-page refcounting. */
-void sh_destroy_shadow(struct vcpu *v, mfn_t smfn);
+void sh_destroy_shadow(struct domain *d, mfn_t smfn);
/* Increase the refcount of a shadow page. Arguments are the mfn to refcount,
* and the physical address of the shadow entry that holds the ref (or zero
@@ -572,9 +572,8 @@ static inline int sh_get_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
/* Decrease the refcount of a shadow page. As for get_ref, takes the
* physical address of the shadow entry that held this reference. */
-static inline void sh_put_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
+static inline void sh_put_ref(struct domain *d, mfn_t smfn, paddr_t entry_pa)
{
- struct domain *d = v->domain;
u32 x, nx;
struct page_info *sp = mfn_to_page(smfn);
@@ -602,7 +601,7 @@ static inline void sh_put_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
sp->u.sh.count = nx;
if ( unlikely(nx == 0) )
- sh_destroy_shadow(v, smfn);
+ sh_destroy_shadow(d, smfn);
}
@@ -728,7 +727,7 @@ static inline void sh_unpin(struct vcpu *v, mfn_t smfn)
}
sh_terminate_list(&tmp_list);
- sh_put_ref(v, smfn, 0);
+ sh_put_ref(d, smfn, 0);
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 11/20] x86/shadow: Alter sh_get_ref() and sh_{, un}pin() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (9 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 10/20] x86/shadow: Alter sh_put_ref() and shadow destroy functions " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 12/20] x86/shadow: Alter shadow_set_l?e() " Andrew Cooper
` (10 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 6 +++---
xen/arch/x86/mm/shadow/multi.c | 16 ++++++++--------
xen/arch/x86/mm/shadow/private.h | 11 ++++-------
3 files changed, 15 insertions(+), 18 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index e2ea6cb..046201a 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1305,7 +1305,7 @@ static void _shadow_prealloc(
/* Unpin this top-level shadow */
trace_shadow_prealloc_unpin(d, smfn);
- sh_unpin(v, smfn);
+ sh_unpin(d, smfn);
/* See if that freed up enough space */
if ( d->arch.paging.shadow.free_pages >= pages ) return;
@@ -1370,7 +1370,7 @@ static void shadow_blow_tables(struct domain *d)
foreach_pinned_shadow(d, sp, t)
{
smfn = page_to_mfn(sp);
- sh_unpin(v, smfn);
+ sh_unpin(d, smfn);
}
/* Second pass: unhook entries of in-use shadows */
@@ -2616,7 +2616,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
break; \
} \
if ( sh_type_is_pinnable(d, t) ) \
- sh_unpin(v, smfn); \
+ sh_unpin(d, smfn); \
else if ( sh_type_has_up_pointer(d, t) ) \
sh_remove_shadow_via_pointer(v, smfn); \
if( !fast \
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 7d82d90..ccb08d3 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -906,10 +906,10 @@ static int shadow_set_l4e(struct vcpu *v,
{
/* About to install a new reference */
mfn_t sl3mfn = shadow_l4e_get_mfn(new_sl4e);
- ok = sh_get_ref(v, sl3mfn, paddr);
+ ok = sh_get_ref(d, sl3mfn, paddr);
/* Are we pinning l3 shadows to handle wierd linux behaviour? */
if ( sh_type_is_pinnable(d, SH_type_l3_64_shadow) )
- ok |= sh_pin(v, sl3mfn);
+ ok |= sh_pin(d, sl3mfn);
if ( !ok )
{
domain_crash(d);
@@ -956,7 +956,7 @@ static int shadow_set_l3e(struct vcpu *v,
if ( shadow_l3e_get_flags(new_sl3e) & _PAGE_PRESENT )
{
/* About to install a new reference */
- if ( !sh_get_ref(v, shadow_l3e_get_mfn(new_sl3e), paddr) )
+ if ( !sh_get_ref(d, shadow_l3e_get_mfn(new_sl3e), paddr) )
{
domain_crash(d);
return SHADOW_SET_ERROR;
@@ -1018,7 +1018,7 @@ static int shadow_set_l2e(struct vcpu *v,
ASSERT(mfn_to_page(sl1mfn)->u.sh.head);
/* About to install a new reference */
- if ( !sh_get_ref(v, sl1mfn, paddr) )
+ if ( !sh_get_ref(d, sl1mfn, paddr) )
{
domain_crash(d);
return SHADOW_SET_ERROR;
@@ -1537,7 +1537,7 @@ sh_make_shadow(struct vcpu *v, mfn_t gmfn, u32 shadow_type)
page_list_for_each_safe(sp, t, &d->arch.paging.shadow.pinned_shadows)
{
if ( sp->u.sh.type == SH_type_l3_64_shadow )
- sh_unpin(v, page_to_mfn(sp));
+ sh_unpin(d, page_to_mfn(sp));
}
d->arch.paging.shadow.opt_flags &= ~SHOPT_LINUX_L3_TOPLEVEL;
sh_reset_l3_up_pointers(v);
@@ -3866,7 +3866,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
ASSERT(mfn_valid(smfn));
/* Pin the shadow and put it (back) on the list of pinned shadows */
- if ( sh_pin(v, smfn) == 0 )
+ if ( sh_pin(d, smfn) == 0 )
{
SHADOW_ERROR("can't pin %#lx as toplevel shadow\n", mfn_x(smfn));
domain_crash(d);
@@ -3874,7 +3874,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
/* Take a ref to this page: it will be released in sh_detach_old_tables()
* or the next call to set_toplevel_shadow() */
- if ( !sh_get_ref(v, smfn, 0) )
+ if ( !sh_get_ref(d, smfn, 0) )
{
SHADOW_ERROR("can't install %#lx as toplevel shadow\n", mfn_x(smfn));
domain_crash(d);
@@ -3895,7 +3895,7 @@ sh_set_toplevel_shadow(struct vcpu *v,
/* Need to repin the old toplevel shadow if it's been unpinned
* by shadow_prealloc(): in PV mode we're still running on this
* shadow and it's not safe to free it yet. */
- if ( !mfn_to_page(old_smfn)->u.sh.pinned && !sh_pin(v, old_smfn) )
+ if ( !mfn_to_page(old_smfn)->u.sh.pinned && !sh_pin(d, old_smfn) )
{
SHADOW_ERROR("can't re-pin %#lx\n", mfn_x(old_smfn));
domain_crash(d);
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index a848c94..cddfde6 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -538,9 +538,8 @@ void sh_destroy_shadow(struct domain *d, mfn_t smfn);
* and the physical address of the shadow entry that holds the ref (or zero
* if the ref is held by something else).
* Returns 0 for failure, 1 for success. */
-static inline int sh_get_ref(struct vcpu *v, mfn_t smfn, paddr_t entry_pa)
+static inline int sh_get_ref(struct domain *d, mfn_t smfn, paddr_t entry_pa)
{
- struct domain *d = v->domain;
u32 x, nx;
struct page_info *sp = mfn_to_page(smfn);
@@ -645,9 +644,8 @@ prev_pinned_shadow(const struct page_info *page,
/* Pin a shadow page: take an extra refcount, set the pin bit,
* and put the shadow at the head of the list of pinned shadows.
* Returns 0 for failure, 1 for success. */
-static inline int sh_pin(struct vcpu *v, mfn_t smfn)
+static inline int sh_pin(struct domain *d, mfn_t smfn)
{
- struct domain *d = v->domain;
struct page_info *sp[4];
struct page_list_head *pin_list;
unsigned int i, pages;
@@ -681,7 +679,7 @@ static inline int sh_pin(struct vcpu *v, mfn_t smfn)
else
{
/* Not pinned: pin it! */
- if ( !sh_get_ref(v, smfn, 0) )
+ if ( !sh_get_ref(d, smfn, 0) )
return 0;
sp[0]->u.sh.pinned = 1;
}
@@ -695,9 +693,8 @@ static inline int sh_pin(struct vcpu *v, mfn_t smfn)
/* Unpin a shadow page: unset the pin bit, take the shadow off the list
* of pinned shadows, and release the extra ref. */
-static inline void sh_unpin(struct vcpu *v, mfn_t smfn)
+static inline void sh_unpin(struct domain *d, mfn_t smfn)
{
- struct domain *d = v->domain;
struct page_list_head tmp_list, *pin_list;
struct page_info *sp, *next;
unsigned int i, head_type;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 12/20] x86/shadow: Alter shadow_set_l?e() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (10 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 11/20] x86/shadow: Alter sh_get_ref() and sh_{, un}pin() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 13/20] x86/shadow: Alter sh_{clear_shadow_entry, remove_shadow_via_pointer}() " Andrew Cooper
` (9 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/multi.c | 69 ++++++++++++++++++++--------------------
1 file changed, 35 insertions(+), 34 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index ccb08d3..1db8161 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -885,12 +885,11 @@ shadow_put_page_from_l1e(shadow_l1e_t sl1e, struct domain *d)
}
#if GUEST_PAGING_LEVELS >= 4
-static int shadow_set_l4e(struct vcpu *v,
+static int shadow_set_l4e(struct domain *d,
shadow_l4e_t *sl4e,
shadow_l4e_t new_sl4e,
mfn_t sl4mfn)
{
- struct domain *d = v->domain;
int flags = 0, ok;
shadow_l4e_t old_sl4e;
paddr_t paddr;
@@ -936,12 +935,11 @@ static int shadow_set_l4e(struct vcpu *v,
return flags;
}
-static int shadow_set_l3e(struct vcpu *v,
+static int shadow_set_l3e(struct domain *d,
shadow_l3e_t *sl3e,
shadow_l3e_t new_sl3e,
mfn_t sl3mfn)
{
- struct domain *d = v->domain;
int flags = 0;
shadow_l3e_t old_sl3e;
paddr_t paddr;
@@ -983,12 +981,11 @@ static int shadow_set_l3e(struct vcpu *v,
}
#endif /* GUEST_PAGING_LEVELS >= 4 */
-static int shadow_set_l2e(struct vcpu *v,
+static int shadow_set_l2e(struct domain *d,
shadow_l2e_t *sl2e,
shadow_l2e_t new_sl2e,
mfn_t sl2mfn)
{
- struct domain *d = v->domain;
int flags = 0;
shadow_l2e_t old_sl2e;
paddr_t paddr;
@@ -1165,14 +1162,13 @@ static inline void shadow_vram_put_l1e(shadow_l1e_t old_sl1e,
}
}
-static int shadow_set_l1e(struct vcpu *v,
+static int shadow_set_l1e(struct domain *d,
shadow_l1e_t *sl1e,
shadow_l1e_t new_sl1e,
p2m_type_t new_type,
mfn_t sl1mfn)
{
int flags = 0;
- struct domain *d = v->domain;
shadow_l1e_t old_sl1e;
#if SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC
mfn_t new_gmfn = shadow_l1e_get_mfn(new_sl1e);
@@ -1699,7 +1695,7 @@ static shadow_l3e_t * shadow_get_and_create_l3e(struct vcpu *v,
}
/* Install the new sl3 table in the sl4e */
l4e_propagate_from_guest(v, gw->l4e, *sl3mfn, &new_sl4e, ft);
- r = shadow_set_l4e(v, sl4e, new_sl4e, sl4mfn);
+ r = shadow_set_l4e(d, sl4e, new_sl4e, sl4mfn);
ASSERT((r & SHADOW_SET_FLUSH) == 0);
if ( r & SHADOW_SET_ERROR )
return NULL;
@@ -1755,7 +1751,7 @@ static shadow_l2e_t * shadow_get_and_create_l2e(struct vcpu *v,
}
/* Install the new sl2 table in the sl3e */
l3e_propagate_from_guest(v, gw->l3e, *sl2mfn, &new_sl3e, ft);
- r = shadow_set_l3e(v, sl3e, new_sl3e, sl3mfn);
+ r = shadow_set_l3e(d, sl3e, new_sl3e, sl3mfn);
ASSERT((r & SHADOW_SET_FLUSH) == 0);
if ( r & SHADOW_SET_ERROR )
return NULL;
@@ -1845,7 +1841,7 @@ static shadow_l1e_t * shadow_get_and_create_l1e(struct vcpu *v,
}
/* Install the new sl1 table in the sl2e */
l2e_propagate_from_guest(v, gw->l2e, *sl1mfn, &new_sl2e, ft);
- r = shadow_set_l2e(v, sl2e, new_sl2e, sl2mfn);
+ r = shadow_set_l2e(d, sl2e, new_sl2e, sl2mfn);
ASSERT((r & SHADOW_SET_FLUSH) == 0);
if ( r & SHADOW_SET_ERROR )
return NULL;
@@ -2084,7 +2080,7 @@ void sh_unhook_32b_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
shadow_l2e_t *sl2e;
SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( !user_only || (sl2e->l2 & _PAGE_USER) )
- (void) shadow_set_l2e(v, sl2e, shadow_l2e_empty(), sl2mfn);
+ (void) shadow_set_l2e(d, sl2e, shadow_l2e_empty(), sl2mfn);
});
}
@@ -2097,7 +2093,7 @@ void sh_unhook_pae_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
shadow_l2e_t *sl2e;
SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( !user_only || (sl2e->l2 & _PAGE_USER) )
- (void) shadow_set_l2e(v, sl2e, shadow_l2e_empty(), sl2mfn);
+ (void) shadow_set_l2e(d, sl2e, shadow_l2e_empty(), sl2mfn);
});
}
@@ -2109,7 +2105,7 @@ void sh_unhook_64b_mappings(struct vcpu *v, mfn_t sl4mfn, int user_only)
shadow_l4e_t *sl4e;
SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, d, {
if ( !user_only || (sl4e->l4 & _PAGE_USER) )
- (void) shadow_set_l4e(v, sl4e, shadow_l4e_empty(), sl4mfn);
+ (void) shadow_set_l4e(d, sl4e, shadow_l4e_empty(), sl4mfn);
});
}
@@ -2180,7 +2176,7 @@ static int validate_gl4e(struct vcpu *v, void *new_ge, mfn_t sl4mfn, void *se)
}
}
- result |= shadow_set_l4e(v, sl4p, new_sl4e, sl4mfn);
+ result |= shadow_set_l4e(d, sl4p, new_sl4e, sl4mfn);
return result;
}
@@ -2212,7 +2208,7 @@ static int validate_gl3e(struct vcpu *v, void *new_ge, mfn_t sl3mfn, void *se)
#endif
}
l3e_propagate_from_guest(v, new_gl3e, sl2mfn, &new_sl3e, ft_prefetch);
- result |= shadow_set_l3e(v, sl3p, new_sl3e, sl3mfn);
+ result |= shadow_set_l3e(d, sl3p, new_sl3e, sl3mfn);
return result;
}
@@ -2259,7 +2255,7 @@ static int validate_gl2e(struct vcpu *v, void *new_ge, mfn_t sl2mfn, void *se)
}
l2e_propagate_from_guest(v, new_gl2e, sl1mfn, &new_sl2e, ft_prefetch);
- result |= shadow_set_l2e(v, sl2p, new_sl2e, sl2mfn);
+ result |= shadow_set_l2e(d, sl2p, new_sl2e, sl2mfn);
return result;
}
@@ -2284,7 +2280,7 @@ static int validate_gl1e(struct vcpu *v, void *new_ge, mfn_t sl1mfn, void *se)
gmfn = get_gfn_query_unlocked(d, gfn_x(gfn), &p2mt);
l1e_propagate_from_guest(v, new_gl1e, gmfn, &new_sl1e, ft_prefetch, p2mt);
- result |= shadow_set_l1e(v, sl1p, new_sl1e, p2mt, sl1mfn);
+ result |= shadow_set_l1e(d, sl1p, new_sl1e, p2mt, sl1mfn);
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
gl1mfn = backpointer(mfn_to_page(sl1mfn));
@@ -2344,7 +2340,7 @@ void sh_resync_l1(struct vcpu *v, mfn_t gl1mfn, mfn_t snpmfn)
gfn = guest_l1e_get_gfn(gl1e);
gmfn = get_gfn_query_unlocked(d, gfn_x(gfn), &p2mt);
l1e_propagate_from_guest(v, gl1e, gmfn, &nsl1e, ft_prefetch, p2mt);
- rc |= shadow_set_l1e(v, sl1p, nsl1e, p2mt, sl1mfn);
+ rc |= shadow_set_l1e(d, sl1p, nsl1e, p2mt, sl1mfn);
*snpl1p = gl1e;
}
});
@@ -2673,7 +2669,7 @@ static void sh_prefetch(struct vcpu *v, walk_t *gw,
/* Propagate the entry. */
l1e_propagate_from_guest(v, gl1e, gmfn, &sl1e, ft_prefetch, p2mt);
- (void) shadow_set_l1e(v, ptr_sl1e + i, sl1e, p2mt, sl1mfn);
+ (void) shadow_set_l1e(d, ptr_sl1e + i, sl1e, p2mt, sl1mfn);
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
if ( snpl1p != NULL )
@@ -3161,7 +3157,7 @@ static int sh_page_fault(struct vcpu *v,
/* Calculate the shadow entry and write it */
l1e_propagate_from_guest(v, gw.l1e, gmfn, &sl1e, ft, p2mt);
- r = shadow_set_l1e(v, ptr_sl1e, sl1e, p2mt, sl1mfn);
+ r = shadow_set_l1e(d, ptr_sl1e, sl1e, p2mt, sl1mfn);
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
if ( mfn_valid(gw.l1mfn)
@@ -3590,7 +3586,7 @@ sh_invlpg(struct vcpu *v, unsigned long va)
shadow_l1e_t *sl1;
sl1 = sh_linear_l1_table(v) + shadow_l1_linear_offset(va);
/* Remove the shadow entry that maps this VA */
- (void) shadow_set_l1e(v, sl1, shadow_l1e_empty(),
+ (void) shadow_set_l1e(d, sl1, shadow_l1e_empty(),
p2m_invalid, sl1mfn);
}
paging_unlock(d);
@@ -4221,7 +4217,7 @@ int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
/* Found it! Need to remove its write permissions. */
sl1e = shadow_l1e_remove_flags(sl1e, _PAGE_RW);
- r = shadow_set_l1e(v, sl1p, sl1e, p2m_ram_rw, smfn);
+ r = shadow_set_l1e(d, sl1p, sl1e, p2m_ram_rw, smfn);
ASSERT( !(r & SHADOW_SET_ERROR) );
sh_unmap_domain_page(sl1p);
@@ -4239,6 +4235,7 @@ static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
/* Look up this vaddr in the current shadow and see if it's a writeable
* mapping of this gmfn. If so, remove it. Returns 1 if it worked. */
{
+ struct domain *d = v->domain;
shadow_l1e_t sl1e, *sl1p;
shadow_l2e_t *sl2p;
shadow_l3e_t *sl3p;
@@ -4275,7 +4272,7 @@ static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
/* Found it! Need to remove its write permissions. */
sl1mfn = shadow_l2e_get_mfn(*sl2p);
sl1e = shadow_l1e_remove_flags(sl1e, _PAGE_RW);
- r = shadow_set_l1e(v, sl1p, sl1e, p2m_ram_rw, sl1mfn);
+ r = shadow_set_l1e(d, sl1p, sl1e, p2m_ram_rw, sl1mfn);
if ( r & SHADOW_SET_ERROR ) {
/* Can only currently happen if we found a grant-mapped
* page. Just make the guess fail. */
@@ -4290,11 +4287,11 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
mfn_t readonly_mfn)
/* Excises all writeable mappings to readonly_mfn from this l1 shadow table */
{
+ struct domain *d = v->domain;
shadow_l1e_t *sl1e;
int done = 0;
int flags;
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
- struct domain *d = v->domain;
struct vcpu *curr = current;
mfn_t base_sl1mfn = sl1mfn; /* Because sl1mfn changes in the foreach */
#endif
@@ -4307,7 +4304,7 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
&& (mfn_x(shadow_l1e_get_mfn(*sl1e)) == mfn_x(readonly_mfn)) )
{
shadow_l1e_t ro_sl1e = shadow_l1e_remove_flags(*sl1e, _PAGE_RW);
- (void) shadow_set_l1e(v, sl1e, ro_sl1e, p2m_ram_rw, sl1mfn);
+ (void) shadow_set_l1e(d, sl1e, ro_sl1e, p2m_ram_rw, sl1mfn);
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
/* Remember the last shadow that we shot a writeable mapping in */
if ( curr->domain == d )
@@ -4326,6 +4323,7 @@ int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
int sh_rm_mappings_from_l1(struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn)
/* Excises all mappings to guest frame from this shadow l1 table */
{
+ struct domain *d = v->domain;
shadow_l1e_t *sl1e;
int done = 0;
int flags;
@@ -4336,7 +4334,7 @@ int sh_rm_mappings_from_l1(struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn)
if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l1e_get_mfn(*sl1e)) == mfn_x(target_mfn)) )
{
- (void) shadow_set_l1e(v, sl1e, shadow_l1e_empty(),
+ (void) shadow_set_l1e(d, sl1e, shadow_l1e_empty(),
p2m_invalid, sl1mfn);
if ( sh_check_page_has_no_refs(mfn_to_page(target_mfn)) )
/* This breaks us cleanly out of the FOREACH macro */
@@ -4352,23 +4350,25 @@ int sh_rm_mappings_from_l1(struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn)
void sh_clear_shadow_entry(struct vcpu *v, void *ep, mfn_t smfn)
/* Blank out a single shadow entry */
{
+ struct domain *d = v->domain;
+
switch ( mfn_to_page(smfn)->u.sh.type )
{
case SH_type_l1_shadow:
- (void) shadow_set_l1e(v, ep, shadow_l1e_empty(), p2m_invalid, smfn);
+ (void) shadow_set_l1e(d, ep, shadow_l1e_empty(), p2m_invalid, smfn);
break;
case SH_type_l2_shadow:
#if GUEST_PAGING_LEVELS >= 3
case SH_type_l2h_shadow:
#endif
- (void) shadow_set_l2e(v, ep, shadow_l2e_empty(), smfn);
+ (void) shadow_set_l2e(d, ep, shadow_l2e_empty(), smfn);
break;
#if GUEST_PAGING_LEVELS >= 4
case SH_type_l3_shadow:
- (void) shadow_set_l3e(v, ep, shadow_l3e_empty(), smfn);
+ (void) shadow_set_l3e(d, ep, shadow_l3e_empty(), smfn);
break;
case SH_type_l4_shadow:
- (void) shadow_set_l4e(v, ep, shadow_l4e_empty(), smfn);
+ (void) shadow_set_l4e(d, ep, shadow_l4e_empty(), smfn);
break;
#endif
default: BUG(); /* Called with the wrong kind of shadow. */
@@ -4389,7 +4389,7 @@ int sh_remove_l1_shadow(struct vcpu *v, mfn_t sl2mfn, mfn_t sl1mfn)
if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l2e_get_mfn(*sl2e)) == mfn_x(sl1mfn)) )
{
- (void) shadow_set_l2e(v, sl2e, shadow_l2e_empty(), sl2mfn);
+ (void) shadow_set_l2e(d, sl2e, shadow_l2e_empty(), sl2mfn);
if ( mfn_to_page(sl1mfn)->u.sh.type == 0 )
/* This breaks us cleanly out of the FOREACH macro */
done = 1;
@@ -4402,6 +4402,7 @@ int sh_remove_l1_shadow(struct vcpu *v, mfn_t sl2mfn, mfn_t sl1mfn)
int sh_remove_l2_shadow(struct vcpu *v, mfn_t sl3mfn, mfn_t sl2mfn)
/* Remove all mappings of this l2 shadow from this l3 shadow */
{
+ struct domain *d = v->domain;
shadow_l3e_t *sl3e;
int done = 0;
int flags;
@@ -4412,7 +4413,7 @@ int sh_remove_l2_shadow(struct vcpu *v, mfn_t sl3mfn, mfn_t sl2mfn)
if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l3e_get_mfn(*sl3e)) == mfn_x(sl2mfn)) )
{
- (void) shadow_set_l3e(v, sl3e, shadow_l3e_empty(), sl3mfn);
+ (void) shadow_set_l3e(d, sl3e, shadow_l3e_empty(), sl3mfn);
if ( mfn_to_page(sl2mfn)->u.sh.type == 0 )
/* This breaks us cleanly out of the FOREACH macro */
done = 1;
@@ -4435,7 +4436,7 @@ int sh_remove_l3_shadow(struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn)
if ( (flags & _PAGE_PRESENT)
&& (mfn_x(shadow_l4e_get_mfn(*sl4e)) == mfn_x(sl3mfn)) )
{
- (void) shadow_set_l4e(v, sl4e, shadow_l4e_empty(), sl4mfn);
+ (void) shadow_set_l4e(d, sl4e, shadow_l4e_empty(), sl4mfn);
if ( mfn_to_page(sl3mfn)->u.sh.type == 0 )
/* This breaks us cleanly out of the FOREACH macro */
done = 1;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 13/20] x86/shadow: Alter sh_{clear_shadow_entry, remove_shadow_via_pointer}() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (11 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 12/20] x86/shadow: Alter shadow_set_l?e() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 14/20] x86/shadow: Alter sh_remove_l?_shadow() " Andrew Cooper
` (8 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 11 +++++------
xen/arch/x86/mm/shadow/multi.c | 4 +---
xen/arch/x86/mm/shadow/multi.h | 2 +-
3 files changed, 7 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 046201a..e522b60 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -2466,11 +2466,10 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
/**************************************************************************/
/* Remove all shadows of a guest frame from the shadow tables */
-static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
+static int sh_remove_shadow_via_pointer(struct domain *d, mfn_t smfn)
/* Follow this shadow's up-pointer, if it has one, and remove the reference
* found there. Returns 1 if that was the only reference to this shadow */
{
- struct domain *d = v->domain;
struct page_info *sp = mfn_to_page(smfn);
mfn_t pmfn;
void *vaddr;
@@ -2496,19 +2495,19 @@ static int sh_remove_shadow_via_pointer(struct vcpu *v, mfn_t smfn)
{
case SH_type_l1_32_shadow:
case SH_type_l2_32_shadow:
- SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, 2)(v, vaddr, pmfn);
+ SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, 2)(d, vaddr, pmfn);
break;
case SH_type_l1_pae_shadow:
case SH_type_l2_pae_shadow:
case SH_type_l2h_pae_shadow:
- SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, 3)(v, vaddr, pmfn);
+ SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, 3)(d, vaddr, pmfn);
break;
case SH_type_l1_64_shadow:
case SH_type_l2_64_shadow:
case SH_type_l2h_64_shadow:
case SH_type_l3_64_shadow:
case SH_type_l4_64_shadow:
- SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, 4)(v, vaddr, pmfn);
+ SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, 4)(d, vaddr, pmfn);
break;
default: BUG(); /* Some wierd unknown shadow type */
}
@@ -2618,7 +2617,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
if ( sh_type_is_pinnable(d, t) ) \
sh_unpin(d, smfn); \
else if ( sh_type_has_up_pointer(d, t) ) \
- sh_remove_shadow_via_pointer(v, smfn); \
+ sh_remove_shadow_via_pointer(d, smfn); \
if( !fast \
&& (pg->count_info & PGC_page_table) \
&& (pg->shadow_flags & (1 << t)) ) \
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 1db8161..469ad25 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -4347,11 +4347,9 @@ int sh_rm_mappings_from_l1(struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn)
/**************************************************************************/
/* Functions to excise all pointers to shadows from higher-level shadows. */
-void sh_clear_shadow_entry(struct vcpu *v, void *ep, mfn_t smfn)
+void sh_clear_shadow_entry(struct domain *d, void *ep, mfn_t smfn)
/* Blank out a single shadow entry */
{
- struct domain *d = v->domain;
-
switch ( mfn_to_page(smfn)->u.sh.type )
{
case SH_type_l1_shadow:
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 614103d..e33948c 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -69,7 +69,7 @@ SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, GUEST_LEVELS)
extern void
SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, GUEST_LEVELS)
- (struct vcpu *v, void *ep, mfn_t smfn);
+ (struct domain *d, void *ep, mfn_t smfn);
extern int
SHADOW_INTERNAL_NAME(sh_remove_l1_shadow, GUEST_LEVELS)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 14/20] x86/shadow: Alter sh_remove_l?_shadow() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (12 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 13/20] x86/shadow: Alter sh_{clear_shadow_entry, remove_shadow_via_pointer}() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 15/20] x86/shadow: Alter shadow_unhook{_???}_mappings() " Andrew Cooper
` (7 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
This involves introducing the domain variant of hash_foreach()
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 52 +++++++++++++++++++++++++++++++++++++--
xen/arch/x86/mm/shadow/multi.c | 9 +++----
xen/arch/x86/mm/shadow/multi.h | 6 ++---
3 files changed, 56 insertions(+), 11 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index e522b60..3810b75 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1998,6 +1998,7 @@ void shadow_hash_delete(struct domain *d, unsigned long n, unsigned int t,
}
typedef int (*hash_vcpu_callback_t)(struct vcpu *v, mfn_t smfn, mfn_t other_mfn);
+typedef int (*hash_domain_callback_t)(struct domain *d, mfn_t smfn, mfn_t other_mfn);
static void hash_vcpu_foreach(struct vcpu *v, unsigned int callback_mask,
const hash_vcpu_callback_t callbacks[],
@@ -2046,6 +2047,53 @@ static void hash_vcpu_foreach(struct vcpu *v, unsigned int callback_mask,
d->arch.paging.shadow.hash_walking = 0;
}
+static void hash_domain_foreach(struct domain *d,
+ unsigned int callback_mask,
+ const hash_domain_callback_t callbacks[],
+ mfn_t callback_mfn)
+/* Walk the hash table looking at the types of the entries and
+ * calling the appropriate callback function for each entry.
+ * The mask determines which shadow types we call back for, and the array
+ * of callbacks tells us which function to call.
+ * Any callback may return non-zero to let us skip the rest of the scan.
+ *
+ * WARNING: Callbacks MUST NOT add or remove hash entries unless they
+ * then return non-zero to terminate the scan. */
+{
+ int i, done = 0;
+ struct page_info *x;
+
+ ASSERT(paging_locked_by_me(d));
+
+ /* Can be called via p2m code &c after shadow teardown. */
+ if ( unlikely(!d->arch.paging.shadow.hash_table) )
+ return;
+
+ /* Say we're here, to stop hash-lookups reordering the chains */
+ ASSERT(d->arch.paging.shadow.hash_walking == 0);
+ d->arch.paging.shadow.hash_walking = 1;
+
+ for ( i = 0; i < SHADOW_HASH_BUCKETS; i++ )
+ {
+ /* WARNING: This is not safe against changes to the hash table.
+ * The callback *must* return non-zero if it has inserted or
+ * deleted anything from the hash (lookups are OK, though). */
+ for ( x = d->arch.paging.shadow.hash_table[i]; x; x = next_shadow(x) )
+ {
+ if ( callback_mask & (1 << x->u.sh.type) )
+ {
+ ASSERT(x->u.sh.type <= 15);
+ ASSERT(callbacks[x->u.sh.type] != NULL);
+ done = callbacks[x->u.sh.type](d, page_to_mfn(x),
+ callback_mfn);
+ if ( done ) break;
+ }
+ }
+ if ( done ) break;
+ }
+ d->arch.paging.shadow.hash_walking = 0;
+}
+
/**************************************************************************/
/* Destroy a shadow page: simple dispatcher to call the per-type destructor
@@ -2537,7 +2585,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
/* Dispatch table for getting per-type functions: each level must
* be called with the function to remove a lower-level shadow. */
- static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
+ static const hash_domain_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
NULL, /* l1_32 */
NULL, /* fl1_32 */
@@ -2621,7 +2669,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
if( !fast \
&& (pg->count_info & PGC_page_table) \
&& (pg->shadow_flags & (1 << t)) ) \
- hash_vcpu_foreach(v, masks[t], callbacks, smfn); \
+ hash_domain_foreach(d, masks[t], callbacks, smfn); \
} while (0)
DO_UNSHADOW(SH_type_l2_32_shadow);
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 469ad25..ab6ebe2 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -4373,10 +4373,9 @@ void sh_clear_shadow_entry(struct domain *d, void *ep, mfn_t smfn)
}
}
-int sh_remove_l1_shadow(struct vcpu *v, mfn_t sl2mfn, mfn_t sl1mfn)
+int sh_remove_l1_shadow(struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn)
/* Remove all mappings of this l1 shadow from this l2 shadow */
{
- struct domain *d = v->domain;
shadow_l2e_t *sl2e;
int done = 0;
int flags;
@@ -4397,10 +4396,9 @@ int sh_remove_l1_shadow(struct vcpu *v, mfn_t sl2mfn, mfn_t sl1mfn)
}
#if GUEST_PAGING_LEVELS >= 4
-int sh_remove_l2_shadow(struct vcpu *v, mfn_t sl3mfn, mfn_t sl2mfn)
+int sh_remove_l2_shadow(struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn)
/* Remove all mappings of this l2 shadow from this l3 shadow */
{
- struct domain *d = v->domain;
shadow_l3e_t *sl3e;
int done = 0;
int flags;
@@ -4420,10 +4418,9 @@ int sh_remove_l2_shadow(struct vcpu *v, mfn_t sl3mfn, mfn_t sl2mfn)
return done;
}
-int sh_remove_l3_shadow(struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn)
+int sh_remove_l3_shadow(struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn)
/* Remove all mappings of this l3 shadow from this l4 shadow */
{
- struct domain *d = v->domain;
shadow_l4e_t *sl4e;
int done = 0;
int flags;
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index e33948c..8bb8ece 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -73,13 +73,13 @@ SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, GUEST_LEVELS)
extern int
SHADOW_INTERNAL_NAME(sh_remove_l1_shadow, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl2mfn, mfn_t sl1mfn);
+ (struct domain *d, mfn_t sl2mfn, mfn_t sl1mfn);
extern int
SHADOW_INTERNAL_NAME(sh_remove_l2_shadow, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl3mfn, mfn_t sl2mfn);
+ (struct domain *d, mfn_t sl3mfn, mfn_t sl2mfn);
extern int
SHADOW_INTERNAL_NAME(sh_remove_l3_shadow, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl4mfn, mfn_t sl3mfn);
+ (struct domain *d, mfn_t sl4mfn, mfn_t sl3mfn);
#if SHADOW_AUDIT & SHADOW_AUDIT_ENTRIES
int
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 15/20] x86/shadow: Alter shadow_unhook{_???}_mappings() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (13 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 14/20] x86/shadow: Alter sh_remove_l?_shadow() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 16/20] x86/shadow: Alter sh_rm_write_access_from_???() " Andrew Cooper
` (6 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 12 ++++++------
xen/arch/x86/mm/shadow/multi.c | 13 +++++--------
xen/arch/x86/mm/shadow/multi.h | 6 +++---
xen/arch/x86/mm/shadow/private.h | 2 +-
4 files changed, 15 insertions(+), 18 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 3810b75..4a9b94b 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1244,20 +1244,20 @@ static unsigned int shadow_min_acceptable_pages(struct domain *d)
/* Dispatcher function: call the per-mode function that will unhook the
* non-Xen mappings in this top-level shadow mfn. With user_only == 1,
* unhooks only the user-mode mappings. */
-void shadow_unhook_mappings(struct vcpu *v, mfn_t smfn, int user_only)
+void shadow_unhook_mappings(struct domain *d, mfn_t smfn, int user_only)
{
struct page_info *sp = mfn_to_page(smfn);
switch ( sp->u.sh.type )
{
case SH_type_l2_32_shadow:
- SHADOW_INTERNAL_NAME(sh_unhook_32b_mappings, 2)(v, smfn, user_only);
+ SHADOW_INTERNAL_NAME(sh_unhook_32b_mappings, 2)(d, smfn, user_only);
break;
case SH_type_l2_pae_shadow:
case SH_type_l2h_pae_shadow:
- SHADOW_INTERNAL_NAME(sh_unhook_pae_mappings, 3)(v, smfn, user_only);
+ SHADOW_INTERNAL_NAME(sh_unhook_pae_mappings, 3)(d, smfn, user_only);
break;
case SH_type_l4_64_shadow:
- SHADOW_INTERNAL_NAME(sh_unhook_64b_mappings, 4)(v, smfn, user_only);
+ SHADOW_INTERNAL_NAME(sh_unhook_64b_mappings, 4)(d, smfn, user_only);
break;
default:
SHADOW_ERROR("top-level shadow has bad type %08x\n", sp->u.sh.type);
@@ -1322,7 +1322,7 @@ static void _shadow_prealloc(
if ( !pagetable_is_null(v2->arch.shadow_table[i]) )
{
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_PREALLOC_UNHOOK);
- shadow_unhook_mappings(v,
+ shadow_unhook_mappings(d,
pagetable_get_mfn(v2->arch.shadow_table[i]), 0);
/* See if that freed up enough space */
@@ -1377,7 +1377,7 @@ static void shadow_blow_tables(struct domain *d)
for_each_vcpu(d, v)
for ( i = 0 ; i < 4 ; i++ )
if ( !pagetable_is_null(v->arch.shadow_table[i]) )
- shadow_unhook_mappings(v,
+ shadow_unhook_mappings(d,
pagetable_get_mfn(v->arch.shadow_table[i]), 0);
/* Make sure everyone sees the unshadowings */
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index ab6ebe2..79d8888 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -2074,9 +2074,8 @@ void sh_destroy_monitor_table(struct vcpu *v, mfn_t mmfn)
#if GUEST_PAGING_LEVELS == 2
-void sh_unhook_32b_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
+void sh_unhook_32b_mappings(struct domain *d, mfn_t sl2mfn, int user_only)
{
- struct domain *d = v->domain;
shadow_l2e_t *sl2e;
SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( !user_only || (sl2e->l2 & _PAGE_USER) )
@@ -2086,10 +2085,9 @@ void sh_unhook_32b_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
#elif GUEST_PAGING_LEVELS == 3
-void sh_unhook_pae_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
+void sh_unhook_pae_mappings(struct domain *d, mfn_t sl2mfn, int user_only)
/* Walk a PAE l2 shadow, unhooking entries from all the subshadows */
{
- struct domain *d = v->domain;
shadow_l2e_t *sl2e;
SHADOW_FOREACH_L2E(sl2mfn, sl2e, 0, 0, d, {
if ( !user_only || (sl2e->l2 & _PAGE_USER) )
@@ -2099,9 +2097,8 @@ void sh_unhook_pae_mappings(struct vcpu *v, mfn_t sl2mfn, int user_only)
#elif GUEST_PAGING_LEVELS == 4
-void sh_unhook_64b_mappings(struct vcpu *v, mfn_t sl4mfn, int user_only)
+void sh_unhook_64b_mappings(struct domain *d, mfn_t sl4mfn, int user_only)
{
- struct domain *d = v->domain;
shadow_l4e_t *sl4e;
SHADOW_FOREACH_L4E(sl4mfn, sl4e, 0, 0, d, {
if ( !user_only || (sl4e->l4 & _PAGE_USER) )
@@ -4506,7 +4503,7 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
{
gmfn = _mfn(mfn_to_page(smfn)->v.sh.back);
mfn_to_page(gmfn)->shadow_flags |= SHF_pagetable_dying;
- shadow_unhook_mappings(v, smfn, 1/* user pages only */);
+ shadow_unhook_mappings(d, smfn, 1/* user pages only */);
flush = 1;
}
}
@@ -4545,7 +4542,7 @@ static void sh_pagetable_dying(struct vcpu *v, paddr_t gpa)
if ( mfn_valid(smfn) )
{
mfn_to_page(gmfn)->shadow_flags |= SHF_pagetable_dying;
- shadow_unhook_mappings(v, smfn, 1/* user pages only */);
+ shadow_unhook_mappings(d, smfn, 1/* user pages only */);
/* Now flush the TLB: we removed toplevel mappings. */
flush_tlb_mask(d->domain_dirty_cpumask);
}
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 8bb8ece..07dffac 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -52,13 +52,13 @@ SHADOW_INTERNAL_NAME(sh_destroy_l4_shadow, GUEST_LEVELS)(
extern void
SHADOW_INTERNAL_NAME(sh_unhook_32b_mappings, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl2mfn, int user_only);
+ (struct domain *d, mfn_t sl2mfn, int user_only);
extern void
SHADOW_INTERNAL_NAME(sh_unhook_pae_mappings, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl3mfn, int user_only);
+ (struct domain *d, mfn_t sl3mfn, int user_only);
extern void
SHADOW_INTERNAL_NAME(sh_unhook_64b_mappings, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl4mfn, int user_only);
+ (struct domain *d, mfn_t sl4mfn, int user_only);
extern int
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, GUEST_LEVELS)
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index cddfde6..14f5d45 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -394,7 +394,7 @@ void shadow_update_paging_modes(struct vcpu *v);
/* Unhook the non-Xen mappings in this top-level shadow mfn.
* With user_only == 1, unhooks only the user-mode mappings. */
-void shadow_unhook_mappings(struct vcpu *v, mfn_t smfn, int user_only);
+void shadow_unhook_mappings(struct domain *d, mfn_t smfn, int user_only);
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
/* Allow a shadowed page to go out of sync */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 16/20] x86/shadow: Alter sh_rm_write_access_from_???() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (14 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 15/20] x86/shadow: Alter shadow_unhook{_???}_mappings() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 17/20] x86/shadow: Alter sh_remove_{all_}shadows{, _and_parents}() " Andrew Cooper
` (5 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 19 ++++++++++---------
xen/arch/x86/mm/shadow/multi.c | 6 ++----
xen/arch/x86/mm/shadow/multi.h | 4 ++--
xen/arch/x86/mm/shadow/private.h | 2 +-
4 files changed, 15 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 4a9b94b..e10b578 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -587,12 +587,13 @@ static inline void _sh_resync_l1(struct vcpu *v, mfn_t gmfn, mfn_t snpmfn)
static inline int oos_fixup_flush_gmfn(struct vcpu *v, mfn_t gmfn,
struct oos_fixup *fixup)
{
+ struct domain *d = v->domain;
int i;
for ( i = 0; i < SHADOW_OOS_FIXUPS; i++ )
{
if ( mfn_x(fixup->smfn[i]) != INVALID_MFN )
{
- sh_remove_write_access_from_sl1p(v, gmfn,
+ sh_remove_write_access_from_sl1p(d, gmfn,
fixup->smfn[i],
fixup->off[i]);
fixup->smfn[i] = _mfn(INVALID_MFN);
@@ -638,7 +639,7 @@ void oos_fixup_add(struct domain *d, mfn_t gmfn,
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_OOS_FIXUP_EVICT);
/* Reuse this slot and remove current writable mapping. */
- sh_remove_write_access_from_sl1p(v, gmfn,
+ sh_remove_write_access_from_sl1p(d, gmfn,
oos_fixup[idx].smfn[next],
oos_fixup[idx].off[next]);
perfc_incr(shadow_oos_fixup_evict);
@@ -2184,7 +2185,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
unsigned long fault_addr)
{
/* Dispatch table for getting per-type functions */
- static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
+ static const hash_domain_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, 2), /* l1_32 */
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, 2), /* fl1_32 */
@@ -2367,7 +2368,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
int shtype = mfn_to_page(last_smfn)->u.sh.type;
if ( callbacks[shtype] )
- callbacks[shtype](curr, last_smfn, gmfn);
+ callbacks[shtype](d, last_smfn, gmfn);
if ( (pg->u.inuse.type_info & PGT_count_mask) != old_count )
perfc_incr(shadow_writeable_h_5);
@@ -2384,7 +2385,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
perfc_incr(shadow_writeable_bf_1);
else
perfc_incr(shadow_writeable_bf);
- hash_vcpu_foreach(v, callback_mask, callbacks, gmfn);
+ hash_domain_foreach(d, callback_mask, callbacks, gmfn);
/* If that didn't catch the mapping, then there's some non-pagetable
* mapping -- ioreq page, grant mapping, &c. */
@@ -2404,7 +2405,7 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
}
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
-int sh_remove_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
+int sh_remove_write_access_from_sl1p(struct domain *d, mfn_t gmfn,
mfn_t smfn, unsigned long off)
{
struct page_info *sp = mfn_to_page(smfn);
@@ -2416,16 +2417,16 @@ int sh_remove_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
|| sp->u.sh.type == SH_type_fl1_32_shadow )
{
return SHADOW_INTERNAL_NAME(sh_rm_write_access_from_sl1p,2)
- (v, gmfn, smfn, off);
+ (d, gmfn, smfn, off);
}
else if ( sp->u.sh.type == SH_type_l1_pae_shadow
|| sp->u.sh.type == SH_type_fl1_pae_shadow )
return SHADOW_INTERNAL_NAME(sh_rm_write_access_from_sl1p,3)
- (v, gmfn, smfn, off);
+ (d, gmfn, smfn, off);
else if ( sp->u.sh.type == SH_type_l1_64_shadow
|| sp->u.sh.type == SH_type_fl1_64_shadow )
return SHADOW_INTERNAL_NAME(sh_rm_write_access_from_sl1p,4)
- (v, gmfn, smfn, off);
+ (d, gmfn, smfn, off);
return 0;
}
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 79d8888..0d1021b 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -4177,10 +4177,9 @@ sh_update_cr3(struct vcpu *v, int do_locking)
/* Functions to revoke guest rights */
#if SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC
-int sh_rm_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
+int sh_rm_write_access_from_sl1p(struct domain *d, mfn_t gmfn,
mfn_t smfn, unsigned long off)
{
- struct domain *d = v->domain;
struct vcpu *curr = current;
int r;
shadow_l1e_t *sl1p, sl1e;
@@ -4280,11 +4279,10 @@ static int sh_guess_wrmap(struct vcpu *v, unsigned long vaddr, mfn_t gmfn)
}
#endif
-int sh_rm_write_access_from_l1(struct vcpu *v, mfn_t sl1mfn,
+int sh_rm_write_access_from_l1(struct domain *d, mfn_t sl1mfn,
mfn_t readonly_mfn)
/* Excises all writeable mappings to readonly_mfn from this l1 shadow table */
{
- struct domain *d = v->domain;
shadow_l1e_t *sl1e;
int done = 0;
int flags;
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 07dffac..1af9225 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -62,7 +62,7 @@ SHADOW_INTERNAL_NAME(sh_unhook_64b_mappings, GUEST_LEVELS)
extern int
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl1mfn, mfn_t readonly_mfn);
+ (struct domain *d, mfn_t sl1mfn, mfn_t readonly_mfn);
extern int
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, GUEST_LEVELS)
(struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn);
@@ -127,5 +127,5 @@ SHADOW_INTERNAL_NAME(sh_safe_not_to_sync, GUEST_LEVELS)
extern int
SHADOW_INTERNAL_NAME(sh_rm_write_access_from_sl1p, GUEST_LEVELS)
- (struct vcpu *v, mfn_t gmfn, mfn_t smfn, unsigned long off);
+ (struct domain *d, mfn_t gmfn, mfn_t smfn, unsigned long off);
#endif
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index 14f5d45..96b53b9 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -405,7 +405,7 @@ void sh_resync(struct domain *d, mfn_t gmfn);
void oos_fixup_add(struct domain *d, mfn_t gmfn, mfn_t smfn, unsigned long off);
-int sh_remove_write_access_from_sl1p(struct vcpu *v, mfn_t gmfn,
+int sh_remove_write_access_from_sl1p(struct domain *d, mfn_t gmfn,
mfn_t smfn, unsigned long offset);
/* Pull all out-of-sync shadows back into sync. If skip != 0, we try
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 17/20] x86/shadow: Alter sh_remove_{all_}shadows{, _and_parents}() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (15 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 16/20] x86/shadow: Alter sh_rm_write_access_from_???() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 18/20] x86/shadow: Alter sh_{remove_all_mappings, rm_mappings_from_l1}() " Andrew Cooper
` (4 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
This allows the removal of 3 improper uses of d->vcpu[0] from toolstack context
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/hvm/hvm.c | 2 +-
xen/arch/x86/mm.c | 4 ++--
xen/arch/x86/mm/shadow/common.c | 18 ++++++++----------
xen/arch/x86/mm/shadow/multi.c | 15 ++++++++-------
xen/include/asm-x86/shadow.h | 8 ++++----
5 files changed, 23 insertions(+), 24 deletions(-)
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index a52c6e0..05d35c0 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -6074,7 +6074,7 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
paging_mark_dirty(d, page_to_mfn(page));
/* These are most probably not page tables any more */
/* don't take a long time and don't die either */
- sh_remove_shadows(d->vcpu[0], _mfn(page_to_mfn(page)), 1, 0);
+ sh_remove_shadows(d, _mfn(page_to_mfn(page)), 1, 0);
put_page(page);
}
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index d4965da..3cc6138 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -2118,7 +2118,7 @@ int free_page_type(struct page_info *page, unsigned long type,
ASSERT(VALID_M2P(gmfn));
/* Page sharing not supported for shadowed domains */
if(!SHARED_M2P(gmfn))
- shadow_remove_all_shadows(owner->vcpu[0], _mfn(gmfn));
+ shadow_remove_all_shadows(owner, _mfn(gmfn));
}
if ( !(type & PGT_partial) )
@@ -2283,7 +2283,7 @@ static int __get_page_type(struct page_info *page, unsigned long type,
&& (page->count_info & PGC_page_table)
&& !((page->shadow_flags & (1u<<29))
&& type == PGT_writable_page) )
- shadow_remove_all_shadows(d->vcpu[0], _mfn(page_to_mfn(page)));
+ shadow_remove_all_shadows(d, _mfn(page_to_mfn(page)));
ASSERT(!(x & PGT_pae_xen_l2));
if ( (x & PGT_type_mask) != type )
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index e10b578..30580ee 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -687,7 +687,7 @@ static int oos_remove_write_access(struct vcpu *v, mfn_t gmfn,
* the page. If that doesn't work either, the guest is granting
* his pagetables and must be killed after all.
* This will flush the tlb, so we can return with no worries. */
- sh_remove_shadows(v, gmfn, 0 /* Be thorough */, 1 /* Must succeed */);
+ sh_remove_shadows(d, gmfn, 0 /* Be thorough */, 1 /* Must succeed */);
return 1;
}
@@ -1130,7 +1130,7 @@ sh_validate_guest_pt_write(struct vcpu *v, mfn_t gmfn,
* Since the validate call above will have made a "safe" (i.e. zero)
* shadow entry, we can let the domain live even if we can't fully
* unshadow the page. */
- sh_remove_shadows(v, gmfn, 0, 0);
+ sh_remove_shadows(d, gmfn, 0, 0);
}
}
@@ -2570,7 +2570,7 @@ static int sh_remove_shadow_via_pointer(struct domain *d, mfn_t smfn)
return rc;
}
-void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
+void sh_remove_shadows(struct domain *d, mfn_t gmfn, int fast, int all)
/* Remove the shadows of this guest page.
* If fast != 0, just try the quick heuristic, which will remove
* at most one reference to each shadow of the page. Otherwise, walk
@@ -2579,7 +2579,6 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
* (all != 0 implies fast == 0)
*/
{
- struct domain *d = v->domain;
struct page_info *pg = mfn_to_page(gmfn);
mfn_t smfn;
unsigned char t;
@@ -2633,8 +2632,7 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
* can be called via put_page_type when we clear a shadow l1e).*/
paging_lock_recursive(d);
- SHADOW_PRINTK("d=%d, v=%d, gmfn=%05lx\n",
- d->domain_id, v->vcpu_id, mfn_x(gmfn));
+ SHADOW_PRINTK("d=%d: gmfn=%lx\n", d->domain_id, mfn_x(gmfn));
/* Bail out now if the page is not shadowed */
if ( (pg->count_info & PGC_page_table) == 0 )
@@ -2703,11 +2701,11 @@ void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all)
}
static void
-sh_remove_all_shadows_and_parents(struct vcpu *v, mfn_t gmfn)
+sh_remove_all_shadows_and_parents(struct domain *d, mfn_t gmfn)
/* Even harsher: this is a HVM page that we thing is no longer a pagetable.
* Unshadow it, and recursively unshadow pages that reference it. */
{
- sh_remove_shadows(v, gmfn, 0, 1);
+ sh_remove_shadows(d, gmfn, 0, 1);
/* XXX TODO:
* Rework this hashtable walker to return a linked-list of all
* the shadows it modified, then do breadth-first recursion
@@ -3384,7 +3382,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
p2m_type_t p2mt = p2m_flags_to_type(l1e_get_flags(*p));
if ( (p2m_is_valid(p2mt) || p2m_is_grant(p2mt)) && mfn_valid(mfn) )
{
- sh_remove_all_shadows_and_parents(v, mfn);
+ sh_remove_all_shadows_and_parents(d, mfn);
if ( sh_remove_all_mappings(v, mfn) )
flush_tlb_mask(d->domain_dirty_cpumask);
}
@@ -3419,7 +3417,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
|| l1e_get_pfn(npte[i]) != mfn_x(omfn) )
{
/* This GFN->MFN mapping has gone away */
- sh_remove_all_shadows_and_parents(v, omfn);
+ sh_remove_all_shadows_and_parents(d, omfn);
if ( sh_remove_all_mappings(v, omfn) )
cpumask_or(&flushmask, &flushmask,
d->domain_dirty_cpumask);
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 0d1021b..7705674 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -2564,7 +2564,7 @@ static inline void check_for_early_unshadow(struct vcpu *v, mfn_t gmfn)
& (SHF_L2_32|SHF_L2_PAE|SHF_L2H_PAE|SHF_L4_64))) )
{
perfc_incr(shadow_early_unshadow);
- sh_remove_shadows(v, gmfn, 1, 0 /* Fast, can fail to unshadow */ );
+ sh_remove_shadows(d, gmfn, 1, 0 /* Fast, can fail to unshadow */ );
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_EARLY_UNSHADOW);
}
v->arch.paging.shadow.last_emulated_mfn_for_unshadow = mfn_x(gmfn);
@@ -3251,7 +3251,7 @@ static int sh_page_fault(struct vcpu *v,
SHADOW_PRINTK("user-mode fault to PT, unshadowing mfn %#lx\n",
mfn_x(gmfn));
perfc_incr(shadow_fault_emulate_failed);
- sh_remove_shadows(v, gmfn, 0 /* thorough */, 1 /* must succeed */);
+ sh_remove_shadows(d, gmfn, 0 /* thorough */, 1 /* must succeed */);
trace_shadow_emulate_other(TRC_SHADOW_EMULATE_UNSHADOW_USER,
va, gfn);
goto done;
@@ -3293,7 +3293,7 @@ static int sh_page_fault(struct vcpu *v,
}
if ( !used )
- sh_remove_shadows(v, gmfn, 1 /* fast */, 0 /* can fail */);
+ sh_remove_shadows(d, gmfn, 1 /* fast */, 0 /* can fail */);
}
/*
@@ -3331,7 +3331,7 @@ static int sh_page_fault(struct vcpu *v,
gdprintk(XENLOG_DEBUG, "write to pagetable during event "
"injection: cr2=%#lx, mfn=%#lx\n",
va, mfn_x(gmfn));
- sh_remove_shadows(v, gmfn, 0 /* thorough */, 1 /* must succeed */);
+ sh_remove_shadows(d, gmfn, 0 /* thorough */, 1 /* must succeed */);
trace_shadow_emulate_other(TRC_SHADOW_EMULATE_UNSHADOW_EVTINJ,
va, gfn);
return EXCRET_fault_fixed;
@@ -3365,7 +3365,7 @@ static int sh_page_fault(struct vcpu *v,
/* If this is actually a page table, then we have a bug, and need
* to support more operations in the emulator. More likely,
* though, this is a hint that this page should not be shadowed. */
- shadow_remove_all_shadows(v, gmfn);
+ shadow_remove_all_shadows(d, gmfn);
trace_shadow_emulate_other(TRC_SHADOW_EMULATE_UNSHADOW_UNHANDLED,
va, gfn);
@@ -4626,6 +4626,7 @@ static void *emulate_map_dest(struct vcpu *v,
u32 bytes,
struct sh_emulate_ctxt *sh_ctxt)
{
+ struct domain *d = v->domain;
void *map = NULL;
sh_ctxt->mfn1 = emulate_gva_to_mfn(v, vaddr, sh_ctxt);
@@ -4647,7 +4648,7 @@ static void *emulate_map_dest(struct vcpu *v,
/* Unaligned writes mean probably this isn't a pagetable */
if ( vaddr & (bytes - 1) )
- sh_remove_shadows(v, sh_ctxt->mfn1, 0, 0 /* Slow, can fail */ );
+ sh_remove_shadows(d, sh_ctxt->mfn1, 0, 0 /* Slow, can fail */ );
if ( likely(((vaddr + bytes - 1) & PAGE_MASK) == (vaddr & PAGE_MASK)) )
{
@@ -4674,7 +4675,7 @@ static void *emulate_map_dest(struct vcpu *v,
MAPPING_SILENT_FAIL : MAPPING_UNHANDLEABLE);
/* Cross-page writes mean probably not a pagetable */
- sh_remove_shadows(v, sh_ctxt->mfn2, 0, 0 /* Slow, can fail */ );
+ sh_remove_shadows(d, sh_ctxt->mfn2, 0, 0 /* Slow, can fail */ );
mfns[0] = mfn_x(sh_ctxt->mfn1);
mfns[1] = mfn_x(sh_ctxt->mfn2);
diff --git a/xen/include/asm-x86/shadow.h b/xen/include/asm-x86/shadow.h
index a4c8776..5b9772d 100644
--- a/xen/include/asm-x86/shadow.h
+++ b/xen/include/asm-x86/shadow.h
@@ -79,7 +79,7 @@ void shadow_teardown(struct domain *d);
/* Call once all of the references to the domain have gone away */
void shadow_final_teardown(struct domain *d);
-void sh_remove_shadows(struct vcpu *v, mfn_t gmfn, int fast, int all);
+void sh_remove_shadows(struct domain *d, mfn_t gmfn, int fast, int all);
/* Discard _all_ mappings from the domain's shadows. */
void shadow_blow_tables_per_domain(struct domain *d);
@@ -93,7 +93,7 @@ void shadow_blow_tables_per_domain(struct domain *d);
#define shadow_track_dirty_vram(d, begin_pfn, nr, bitmap) \
({ ASSERT_UNREACHABLE(); -EOPNOTSUPP; })
-static inline void sh_remove_shadows(struct vcpu *v, mfn_t gmfn,
+static inline void sh_remove_shadows(struct domain *d, mfn_t gmfn,
bool_t fast, bool_t all) {}
static inline void shadow_blow_tables_per_domain(struct domain *d) {}
@@ -107,10 +107,10 @@ static inline int shadow_domctl(struct domain *d, xen_domctl_shadow_op_t *sc,
#endif /* CONFIG_SHADOW_PAGING */
/* Remove all shadows of the guest mfn. */
-static inline void shadow_remove_all_shadows(struct vcpu *v, mfn_t gmfn)
+static inline void shadow_remove_all_shadows(struct domain *d, mfn_t gmfn)
{
/* See the comment about locking in sh_remove_shadows */
- sh_remove_shadows(v, gmfn, 0 /* Be thorough */, 1 /* Must succeed */);
+ sh_remove_shadows(d, gmfn, 0 /* Be thorough */, 1 /* Must succeed */);
}
#endif /* _XEN_SHADOW_H */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 18/20] x86/shadow: Alter sh_{remove_all_mappings, rm_mappings_from_l1}() to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (16 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 17/20] x86/shadow: Alter sh_remove_{all_}shadows{, _and_parents}() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 19/20] x86/shadow: Alter sh_remove_write_access " Andrew Cooper
` (3 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
This allows the removal an improper use of d->vcpu[0] from toolstack context
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 13 ++++++-------
xen/arch/x86/mm/shadow/multi.c | 3 +--
xen/arch/x86/mm/shadow/multi.h | 2 +-
3 files changed, 8 insertions(+), 10 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 30580ee..d24859e 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -2436,13 +2436,12 @@ int sh_remove_write_access_from_sl1p(struct domain *d, mfn_t gmfn,
/* Remove all mappings of a guest frame from the shadow tables.
* Returns non-zero if we need to flush TLBs. */
-static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
+static int sh_remove_all_mappings(struct domain *d, mfn_t gmfn)
{
- struct domain *d = v->domain;
struct page_info *page = mfn_to_page(gmfn);
/* Dispatch table for getting per-type functions */
- static const hash_vcpu_callback_t callbacks[SH_type_unused] = {
+ static const hash_domain_callback_t callbacks[SH_type_unused] = {
NULL, /* none */
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, 2), /* l1_32 */
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, 2), /* fl1_32 */
@@ -2484,7 +2483,7 @@ static int sh_remove_all_mappings(struct vcpu *v, mfn_t gmfn)
/* Brute-force search of all the shadows, by walking the hash */
perfc_incr(shadow_mappings_bf);
- hash_vcpu_foreach(v, callback_mask, callbacks, gmfn);
+ hash_domain_foreach(d, callback_mask, callbacks, gmfn);
/* If that didn't catch the mapping, something is very wrong */
if ( !sh_check_page_has_no_refs(page) )
@@ -3383,7 +3382,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
if ( (p2m_is_valid(p2mt) || p2m_is_grant(p2mt)) && mfn_valid(mfn) )
{
sh_remove_all_shadows_and_parents(d, mfn);
- if ( sh_remove_all_mappings(v, mfn) )
+ if ( sh_remove_all_mappings(d, mfn) )
flush_tlb_mask(d->domain_dirty_cpumask);
}
}
@@ -3418,7 +3417,7 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
{
/* This GFN->MFN mapping has gone away */
sh_remove_all_shadows_and_parents(d, omfn);
- if ( sh_remove_all_mappings(v, omfn) )
+ if ( sh_remove_all_mappings(d, omfn) )
cpumask_or(&flushmask, &flushmask,
d->domain_dirty_cpumask);
}
@@ -3634,7 +3633,7 @@ int shadow_track_dirty_vram(struct domain *d,
dirty = 1;
/* TODO: Heuristics for finding the single mapping of
* this gmfn */
- flush_tlb |= sh_remove_all_mappings(d->vcpu[0], mfn);
+ flush_tlb |= sh_remove_all_mappings(d, mfn);
}
else
{
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 7705674..288c7d5 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -4315,10 +4315,9 @@ int sh_rm_write_access_from_l1(struct domain *d, mfn_t sl1mfn,
}
-int sh_rm_mappings_from_l1(struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn)
+int sh_rm_mappings_from_l1(struct domain *d, mfn_t sl1mfn, mfn_t target_mfn)
/* Excises all mappings to guest frame from this shadow l1 table */
{
- struct domain *d = v->domain;
shadow_l1e_t *sl1e;
int done = 0;
int flags;
diff --git a/xen/arch/x86/mm/shadow/multi.h b/xen/arch/x86/mm/shadow/multi.h
index 1af9225..935e12d 100644
--- a/xen/arch/x86/mm/shadow/multi.h
+++ b/xen/arch/x86/mm/shadow/multi.h
@@ -65,7 +65,7 @@ SHADOW_INTERNAL_NAME(sh_rm_write_access_from_l1, GUEST_LEVELS)
(struct domain *d, mfn_t sl1mfn, mfn_t readonly_mfn);
extern int
SHADOW_INTERNAL_NAME(sh_rm_mappings_from_l1, GUEST_LEVELS)
- (struct vcpu *v, mfn_t sl1mfn, mfn_t target_mfn);
+ (struct domain *d, mfn_t sl1mfn, mfn_t target_mfn);
extern void
SHADOW_INTERNAL_NAME(sh_clear_shadow_entry, GUEST_LEVELS)
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 19/20] x86/shadow: Alter sh_remove_write_access to take a domain
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (17 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 18/20] x86/shadow: Alter sh_{remove_all_mappings, rm_mappings_from_l1}() " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-12 17:16 ` [PATCH 20/20] x86/shadow: Cleanup of vcpu handling Andrew Cooper
` (2 subsequent siblings)
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
This allows the removal an improper use of d->vcpu[0] from toolstack context
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 7 +++----
xen/arch/x86/mm/shadow/multi.c | 16 ++++++----------
xen/arch/x86/mm/shadow/private.h | 2 +-
3 files changed, 10 insertions(+), 15 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index d24859e..4e6397a 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -671,7 +671,7 @@ static int oos_remove_write_access(struct vcpu *v, mfn_t gmfn,
ftlb |= oos_fixup_flush_gmfn(v, gmfn, fixup);
- switch ( sh_remove_write_access(v, gmfn, 0, 0) )
+ switch ( sh_remove_write_access(d, gmfn, 0, 0) )
{
default:
case 0:
@@ -2180,7 +2180,7 @@ static inline void trace_shadow_wrmap_bf(mfn_t gmfn)
* level==0 means we have some other reason for revoking write access.
* If level==0 we are allowed to fail, returning -1. */
-int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
+int sh_remove_write_access(struct domain *d, mfn_t gmfn,
unsigned int level,
unsigned long fault_addr)
{
@@ -2212,7 +2212,6 @@ int sh_remove_write_access(struct vcpu *v, mfn_t gmfn,
| SHF_L1_64
| SHF_FL1_64
;
- struct domain *d = v->domain;
struct page_info *pg = mfn_to_page(gmfn);
#if SHADOW_OPTIMIZATIONS & SHOPT_WRITABLE_HEURISTIC
struct vcpu *curr = current;
@@ -3689,7 +3688,7 @@ int shadow_track_dirty_vram(struct domain *d,
for ( i = begin_pfn; i < end_pfn; i++ ) {
mfn_t mfn = get_gfn_query_unlocked(d, i, &t);
if (mfn_x(mfn) != INVALID_MFN)
- flush_tlb |= sh_remove_write_access(d->vcpu[0], mfn, 1, 0);
+ flush_tlb |= sh_remove_write_access(d, mfn, 1, 0);
}
dirty_vram->last_dirty = -1;
}
diff --git a/xen/arch/x86/mm/shadow/multi.c b/xen/arch/x86/mm/shadow/multi.c
index 288c7d5..16cd60d 100644
--- a/xen/arch/x86/mm/shadow/multi.c
+++ b/xen/arch/x86/mm/shadow/multi.c
@@ -278,11 +278,7 @@ shadow_check_gl1e(struct vcpu *v, walk_t *gw)
static inline uint32_t
gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
{
-#if GUEST_PAGING_LEVELS >= 3 /* PAE or 64... */
-#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
struct domain *d = v->domain;
-#endif
-#endif
uint32_t rc = 0;
#if GUEST_PAGING_LEVELS >= 3 /* PAE or 64... */
@@ -295,7 +291,7 @@ gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
}
else
#endif /* OOS */
- if ( sh_remove_write_access(v, gw->l3mfn, 3, va) )
+ if ( sh_remove_write_access(d, gw->l3mfn, 3, va) )
rc = GW_RMWR_FLUSHTLB;
#endif /* GUEST_PAGING_LEVELS >= 4 */
@@ -307,7 +303,7 @@ gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
}
else
#endif /* OOS */
- if ( sh_remove_write_access(v, gw->l2mfn, 2, va) )
+ if ( sh_remove_write_access(d, gw->l2mfn, 2, va) )
rc |= GW_RMWR_FLUSHTLB;
#endif /* GUEST_PAGING_LEVELS >= 3 */
@@ -316,7 +312,7 @@ gw_remove_write_accesses(struct vcpu *v, unsigned long va, walk_t *gw)
#if (SHADOW_OPTIMIZATIONS & SHOPT_OUT_OF_SYNC)
&& !mfn_is_out_of_sync(gw->l1mfn)
#endif /* OOS */
- && sh_remove_write_access(v, gw->l1mfn, 1, va) )
+ && sh_remove_write_access(d, gw->l1mfn, 1, va) )
rc |= GW_RMWR_FLUSHTLB;
return rc;
@@ -4028,7 +4024,7 @@ sh_update_cr3(struct vcpu *v, int do_locking)
* replace the old shadow pagetable(s), so that we can safely use the
* (old) shadow linear maps in the writeable mapping heuristics. */
#if GUEST_PAGING_LEVELS == 2
- if ( sh_remove_write_access(v, gmfn, 2, 0) != 0 )
+ if ( sh_remove_write_access(d, gmfn, 2, 0) != 0 )
flush_tlb_mask(d->domain_dirty_cpumask);
sh_set_toplevel_shadow(v, 0, gmfn, SH_type_l2_shadow);
#elif GUEST_PAGING_LEVELS == 3
@@ -4048,7 +4044,7 @@ sh_update_cr3(struct vcpu *v, int do_locking)
gl2gfn = guest_l3e_get_gfn(gl3e[i]);
gl2mfn = get_gfn_query_unlocked(d, gfn_x(gl2gfn), &p2mt);
if ( p2m_is_ram(p2mt) )
- flush |= sh_remove_write_access(v, gl2mfn, 2, 0);
+ flush |= sh_remove_write_access(d, gl2mfn, 2, 0);
}
}
if ( flush )
@@ -4072,7 +4068,7 @@ sh_update_cr3(struct vcpu *v, int do_locking)
}
}
#elif GUEST_PAGING_LEVELS == 4
- if ( sh_remove_write_access(v, gmfn, 4, 0) != 0 )
+ if ( sh_remove_write_access(d, gmfn, 4, 0) != 0 )
flush_tlb_mask(d->domain_dirty_cpumask);
sh_set_toplevel_shadow(v, 0, gmfn, SH_type_l4_shadow);
#else
diff --git a/xen/arch/x86/mm/shadow/private.h b/xen/arch/x86/mm/shadow/private.h
index 96b53b9..1bf1deb 100644
--- a/xen/arch/x86/mm/shadow/private.h
+++ b/xen/arch/x86/mm/shadow/private.h
@@ -374,7 +374,7 @@ void sh_validate_guest_pt_write(struct vcpu *v, mfn_t gmfn,
* Returns non-zero if we need to flush TLBs.
* level and fault_addr desribe how we found this to be a pagetable;
* level==0 means we have some other reason for revoking write access. */
-extern int sh_remove_write_access(struct vcpu *v, mfn_t readonly_mfn,
+extern int sh_remove_write_access(struct domain *d, mfn_t readonly_mfn,
unsigned int level,
unsigned long fault_addr);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH 20/20] x86/shadow: Cleanup of vcpu handling
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (18 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 19/20] x86/shadow: Alter sh_remove_write_access " Andrew Cooper
@ 2015-02-12 17:16 ` Andrew Cooper
2015-02-13 11:42 ` [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
2015-02-16 12:29 ` Tim Deegan
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-12 17:16 UTC (permalink / raw)
To: Xen-devel; +Cc: Andrew Cooper, Tim Deegan, Jan Beulich
There existed some codepaths which had a domain in their hand, but needed a
vcpu to drive the shadow interface.
Now that these interfaces have changed to be domain based, the hoop-jumping
can be removed.
Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Tim Deegan <tim@xen.org>
---
xen/arch/x86/mm/shadow/common.c | 30 ++++++++++--------------------
1 file changed, 10 insertions(+), 20 deletions(-)
diff --git a/xen/arch/x86/mm/shadow/common.c b/xen/arch/x86/mm/shadow/common.c
index 4e6397a..01bc750 100644
--- a/xen/arch/x86/mm/shadow/common.c
+++ b/xen/arch/x86/mm/shadow/common.c
@@ -1280,23 +1280,17 @@ static inline void trace_shadow_prealloc_unpin(struct domain *d, mfn_t smfn)
/* Make sure there are at least count order-sized pages
* available in the shadow page pool. */
-static void _shadow_prealloc(
- struct domain *d,
- unsigned int pages)
+static void _shadow_prealloc(struct domain *d, unsigned int pages)
{
- /* Need a vpcu for calling unpins; for now, since we don't have
- * per-vcpu shadows, any will do */
- struct vcpu *v, *v2;
+ struct vcpu *v;
struct page_info *sp, *t;
mfn_t smfn;
int i;
if ( d->arch.paging.shadow.free_pages >= pages ) return;
- v = current;
- if ( v->domain != d )
- v = d->vcpu[0];
- ASSERT(v != NULL); /* Shouldn't have enabled shadows if we've no vcpus */
+ /* Shouldn't have enabled shadows if we've no vcpus. */
+ ASSERT(d->vcpu && d->vcpu[0]);
/* Stage one: walk the list of pinned pages, unpinning them */
perfc_incr(shadow_prealloc_1);
@@ -1317,14 +1311,14 @@ static void _shadow_prealloc(
* mappings. */
perfc_incr(shadow_prealloc_2);
- for_each_vcpu(d, v2)
+ for_each_vcpu(d, v)
for ( i = 0 ; i < 4 ; i++ )
{
- if ( !pagetable_is_null(v2->arch.shadow_table[i]) )
+ if ( !pagetable_is_null(v->arch.shadow_table[i]) )
{
TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_PREALLOC_UNHOOK);
shadow_unhook_mappings(d,
- pagetable_get_mfn(v2->arch.shadow_table[i]), 0);
+ pagetable_get_mfn(v->arch.shadow_table[i]), 0);
/* See if that freed up enough space */
if ( d->arch.paging.shadow.free_pages >= pages )
@@ -1361,11 +1355,12 @@ void shadow_prealloc(struct domain *d, u32 type, unsigned int count)
static void shadow_blow_tables(struct domain *d)
{
struct page_info *sp, *t;
- struct vcpu *v = d->vcpu[0];
+ struct vcpu *v;
mfn_t smfn;
int i;
- ASSERT(v != NULL);
+ /* Shouldn't have enabled shadows if we've no vcpus. */
+ ASSERT(d->vcpu && d->vcpu[0]);
/* Pass one: unpin all pinned pages */
foreach_pinned_shadow(d, sp, t)
@@ -3363,11 +3358,6 @@ static void sh_unshadow_for_p2m_change(struct domain *d, unsigned long gfn,
l1_pgentry_t *p, l1_pgentry_t new,
unsigned int level)
{
- struct vcpu *v = current;
-
- if ( v->domain != d )
- v = d->vcpu ? d->vcpu[0] : NULL;
-
/* The following assertion is to make sure we don't step on 1GB host
* page support of HVM guest. */
ASSERT(!(level > 2 && (l1e_get_flags(*p) & _PAGE_PRESENT) &&
--
1.7.10.4
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH RFC 00/20] Change parts of the shadow interface to be domain based
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (19 preceding siblings ...)
2015-02-12 17:16 ` [PATCH 20/20] x86/shadow: Cleanup of vcpu handling Andrew Cooper
@ 2015-02-13 11:42 ` Andrew Cooper
2015-02-16 12:29 ` Tim Deegan
21 siblings, 0 replies; 29+ messages in thread
From: Andrew Cooper @ 2015-02-13 11:42 UTC (permalink / raw)
To: Xen-devel; +Cc: Tim Deegan, Keir Fraser, Jan Beulich
On 12/02/15 17:15, Andrew Cooper wrote:
> The entire series has been compile tested at each changeset, for both
> shadow-paging=y and n. It has also been tested internally in XenServer, but
> appears to have gotten caught up in some collateral damage from an unrelated
> merge. I am rerunning the tests.
Reruns of the tests were all fine.
~Andrew
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH RFC 00/20] Change parts of the shadow interface to be domain based
2015-02-12 17:15 [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
` (20 preceding siblings ...)
2015-02-13 11:42 ` [PATCH RFC 00/20] Change parts of the shadow interface to be domain based Andrew Cooper
@ 2015-02-16 12:29 ` Tim Deegan
2015-02-16 13:35 ` Andrew Cooper
21 siblings, 1 reply; 29+ messages in thread
From: Tim Deegan @ 2015-02-16 12:29 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Keir Fraser, Jan Beulich, Xen-devel
Hi,
Thanks for this; I think the code looks a lot saner with all the
vcpu<->domain fiddling removed. :)
At 17:15 +0000 on 12 Feb (1423757759), Andrew Cooper wrote:
> The purpose of this series is to prevent toolstack entry points into the
> shadow code from passing d->vcpu[0] for actions which are inherenly domain
> wide. It also fixes the fact that shadow heuristics were being applied to
> vcpu 0 for toolstack-initiated actions.
>
> This series is composed mostly of mechanical changes. The only patches which
> have a practical difference on shadow execution are patches 4 and 20
So having read the series, you can add my Reviewed-by: to all except 4
and 20. 20 looks probably OK but I want to give it a second pass. 4
definitely needs some thought -- in particular v == current is not the
same as d == current->domain. I wonder if there's some better test;
maybe also checking that current is in the right mode will help. Let
me think about it and I'll review again later in the week. I'll want
to have a look at the remaining users of the vcpu variant hash_foreach
as well.
Cheers,
Tim.
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH RFC 00/20] Change parts of the shadow interface to be domain based
2015-02-16 12:29 ` Tim Deegan
@ 2015-02-16 13:35 ` Andrew Cooper
2015-02-16 18:28 ` Tim Deegan
0 siblings, 1 reply; 29+ messages in thread
From: Andrew Cooper @ 2015-02-16 13:35 UTC (permalink / raw)
To: Tim Deegan; +Cc: Keir Fraser, Jan Beulich, Xen-devel
On 16/02/15 12:29, Tim Deegan wrote:
> Hi,
>
> Thanks for this; I think the code looks a lot saner with all the
> vcpu<->domain fiddling removed. :)
>
> At 17:15 +0000 on 12 Feb (1423757759), Andrew Cooper wrote:
>> The purpose of this series is to prevent toolstack entry points into the
>> shadow code from passing d->vcpu[0] for actions which are inherenly domain
>> wide. It also fixes the fact that shadow heuristics were being applied to
>> vcpu 0 for toolstack-initiated actions.
>>
>> This series is composed mostly of mechanical changes. The only patches which
>> have a practical difference on shadow execution are patches 4 and 20
> So having read the series, you can add my Reviewed-by: to all except 4
> and 20. 20 looks probably OK but I want to give it a second pass.
20 is no practical change. All the patched functions were ones which
had a d in their hand and had to juggle for a v to use the shadow API.
The affected APIs now take d's instead of v's which means the v juggling
can be removed.
> 4 definitely needs some thought -- in particular v == current is not the
> same as d == current->domain. I wonder if there's some better test;
> maybe also checking that current is in the right mode will help.
The result is still the same in that you can derive a valid v when
running in the context of the target d.
The change specifically causes codepaths running in the context of the
toolstack domain to not perform shadow shootdown heuristics.
(It was you who suggested this, although I believe that we were down the
pub at the time ;) )
> Let me think about it and I'll review again later in the week. I'll want
> to have a look at the remaining users of the vcpu variant hash_foreach
> as well.
There are certainly more codepaths which could be turned to taking a
domain. I limited this series just to adjusting the toolstack
entrypoints into the shadow code.
There are several codepaths are almost completely domain generic, other
than needing to check whether a vcpu has EFER.NX set or not, which
causes any codepath which creates shadows to still be vcpu specific.
~Andrew
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH RFC 00/20] Change parts of the shadow interface to be domain based
2015-02-16 13:35 ` Andrew Cooper
@ 2015-02-16 18:28 ` Tim Deegan
2015-02-19 12:35 ` Tim Deegan
0 siblings, 1 reply; 29+ messages in thread
From: Tim Deegan @ 2015-02-16 18:28 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Keir Fraser, Jan Beulich, Xen-devel
At 13:35 +0000 on 16 Feb (1424090147), Andrew Cooper wrote:
> On 16/02/15 12:29, Tim Deegan wrote:
> > Hi,
> >
> > Thanks for this; I think the code looks a lot saner with all the
> > vcpu<->domain fiddling removed. :)
> >
> > At 17:15 +0000 on 12 Feb (1423757759), Andrew Cooper wrote:
> >> The purpose of this series is to prevent toolstack entry points into the
> >> shadow code from passing d->vcpu[0] for actions which are inherenly domain
> >> wide. It also fixes the fact that shadow heuristics were being applied to
> >> vcpu 0 for toolstack-initiated actions.
> >>
> >> This series is composed mostly of mechanical changes. The only patches which
> >> have a practical difference on shadow execution are patches 4 and 20
> > So having read the series, you can add my Reviewed-by: to all except 4
> > and 20. 20 looks probably OK but I want to give it a second pass.
>
> 20 is no practical change.
It is if it's correct. :)
> > 4 definitely needs some thought -- in particular v == current is not the
> > same as d == current->domain. I wonder if there's some better test;
> > maybe also checking that current is in the right mode will help.
>
> The result is still the same in that you can derive a valid v when
> running in the context of the target d.
It's different if there's any path in the existing code where these
functions can be called from another vcpu of the same guest. I don't
_think_ there is such a path, though.
> The change specifically causes codepaths running in the context of the
> toolstack domain to not perform shadow shootdown heuristics.
>
> (It was you who suggested this, although I believe that we were down the
> pub at the time ;) )
Very likely. :) Let me have another look at it later in the week.
> > Let me think about it and I'll review again later in the week. I'll want
> > to have a look at the remaining users of the vcpu variant hash_foreach
> > as well.
>
> There are certainly more codepaths which could be turned to taking a
> domain. I limited this series just to adjusting the toolstack
> entrypoints into the shadow code.
Righto.
Cheers,
Tim.
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH RFC 00/20] Change parts of the shadow interface to be domain based
2015-02-16 18:28 ` Tim Deegan
@ 2015-02-19 12:35 ` Tim Deegan
0 siblings, 0 replies; 29+ messages in thread
From: Tim Deegan @ 2015-02-19 12:35 UTC (permalink / raw)
To: Andrew Cooper; +Cc: Keir Fraser, Jan Beulich, Xen-devel
At 19:28 +0100 on 16 Feb (1424111337), Tim Deegan wrote:
> At 13:35 +0000 on 16 Feb (1424090147), Andrew Cooper wrote:
> > On 16/02/15 12:29, Tim Deegan wrote:
> > > So having read the series, you can add my Reviewed-by: to all except 4
> > > and 20. 20 looks probably OK but I want to give it a second pass.
> >
> > 20 is no practical change.
>
> It is if it's correct. :)
Which it does seem to be. Reviewed-by: Tim Deegan <tim@xen.org>
Cheers,
Tim.
^ permalink raw reply [flat|nested] 29+ messages in thread