[Qemu-devel] the arm cache coherency cluster "v2"

[Qemu-devel] the arm cache coherency cluster "v2"

[Andrew Jones]

In reply to this message I'll send two series' one for KVM and
one for QEMU. The two series' are their respective component
complements, and attempt to implement cache coherency for arm
guests using emulated devices, where the emulator (qemu) uses
cached memory for the device memory, but the guest uses
uncached - as device memory is generally used. Right now I've
just focused on VGA vram.

This approach is the "MADV_UNCACHED" type that Paolo suggested.
This type of approach could also be described as "make userspace's
memory access type match the expected access type of the guest",
and Mario has suggested using a memory driver, which could have
the same result.

The coming series' is inspired by both Paolo's and Mario's
suggestions, but it uses a kvm memslot flag, rather than an
madvise flag, and thus for the memory driver, it's just KVM.

See the thread

https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg01254.html

for some more background.

Thanks in advance for comments.

drew

[Qemu-devel] [RFC PATCH 0/3] KVM: Introduce KVM_MEM_UNCACHED

[Andrew Jones]

Introduce a new memory region flag, KVM_MEM_UNCACHED, which
is needed by ARM. This flag informs KVM that the given memory
region is typically mapped by the guest as uncached. KVM for
ARM then maps that region as uncached for userspace as well,
in order to keep coherency.

Andrew Jones (3):
  KVM: promote KVM_MEMSLOT_INCOHERENT to uapi
  arm/arm64: KVM: decouple READONLY and UNCACHED
  arm/arm64: KVM: implement KVM_MEM_UNCACHED

 Documentation/virtual/kvm/api.txt | 16 ++++---
 arch/arm/include/asm/kvm_mmu.h    |  9 ++++
 arch/arm/include/uapi/asm/kvm.h   |  2 +
 arch/arm/kvm/arm.c                |  1 +
 arch/arm/kvm/mmu.c                | 90 ++++++++++++++++++++++++++++++++++-----
 arch/arm64/include/asm/kvm_mmu.h  |  9 ++++
 arch/arm64/include/uapi/asm/kvm.h |  2 +
 include/linux/kvm_host.h          |  1 -
 include/uapi/linux/kvm.h          |  2 +
 virt/kvm/kvm_main.c               |  7 ++-
 10 files changed, 121 insertions(+), 18 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 1/3] KVM: promote KVM_MEMSLOT_INCOHERENT to uapi

[Andrew Jones]

Also rename to KVM_MEM_UNCACHED.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 Documentation/virtual/kvm/api.txt | 16 ++++++++++------
 arch/arm/include/uapi/asm/kvm.h   |  1 +
 arch/arm/kvm/arm.c                |  1 +
 arch/arm/kvm/mmu.c                |  4 ++--
 arch/arm64/include/uapi/asm/kvm.h |  1 +
 include/linux/kvm_host.h          |  1 -
 include/uapi/linux/kvm.h          |  2 ++
 virt/kvm/kvm_main.c               |  7 ++++++-
 8 files changed, 23 insertions(+), 10 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 0007fef4ed814..a5a51403a7937 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -902,6 +902,7 @@ struct kvm_userspace_memory_region {
 /* for kvm_memory_region::flags */
 #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
 #define KVM_MEM_READONLY	(1UL << 1)
+#define KVM_MEM_UNCACHED	(1UL << 2)
 
 This ioctl allows the user to create or modify a guest physical memory
 slot.  When changing an existing slot, it may be moved in the guest
@@ -917,12 +918,15 @@ It is recommended that the lower 21 bits of guest_phys_addr and userspace_addr
 be identical.  This allows large pages in the guest to be backed by large
 pages in the host.
 
-The flags field supports two flags: KVM_MEM_LOG_DIRTY_PAGES and
-KVM_MEM_READONLY.  The former can be set to instruct KVM to keep track of
-writes to memory within the slot.  See KVM_GET_DIRTY_LOG ioctl to know how to
-use it.  The latter can be set, if KVM_CAP_READONLY_MEM capability allows it,
-to make a new slot read-only.  In this case, writes to this memory will be
-posted to userspace as KVM_EXIT_MMIO exits.
+The flags field supports three flags: KVM_MEM_LOG_DIRTY_PAGES,
+KVM_MEM_READONLY, and KVM_MEM_UNCACHED.  The first can be set to instruct
+KVM to keep track of writes to memory within the slot.  See KVM_GET_DIRTY_LOG
+ioctl to know how to use it.  The second can be set, if KVM_CAP_READONLY_MEM
+capability allows it, to make a new slot read-only.  In this case, writes to
+this memory will be posted to userspace as KVM_EXIT_MMIO exits. The third can
+be set, if the KVM_CAP_UNCACHED_MEM capability allows it. This remaps the
+memory as uncached, i.e. userspace will always directly read/write RAM for
+this memory region.
 
 When the KVM_CAP_SYNC_MMU capability is available, changes in the backing of
 the memory region are automatically reflected into the guest.  For example, an
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 09ee408c1a676..9d6fc19acf8a2 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -26,6 +26,7 @@
 #define __KVM_HAVE_GUEST_DEBUG
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_UNCACHED_MEM
 
 #define KVM_REG_SIZE(id)						\
 	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index c9e6ef1f7403a..8d4c08f238cf0 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -180,6 +180,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)
 	case KVM_CAP_ARM_PSCI:
 	case KVM_CAP_ARM_PSCI_0_2:
 	case KVM_CAP_READONLY_MEM:
+	case KVM_CAP_UNCACHED_MEM:
 		r = 1;
 		break;
 	case KVM_CAP_COALESCED_MMIO:
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index bcc1b3ad2adce..719931e83c468 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1260,7 +1260,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	if (!hugetlb && !force_pte)
 		hugetlb = transparent_hugepage_adjust(&pfn, &fault_ipa);
 
-	fault_ipa_uncached = memslot->flags & KVM_MEMSLOT_INCOHERENT;
+	fault_ipa_uncached = memslot->flags & KVM_MEM_UNCACHED;
 
 	if (hugetlb) {
 		pmd_t new_pmd = pfn_pmd(pfn, mem_type);
@@ -1792,7 +1792,7 @@ int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 	 * regions as incoherent.
 	 */
 	if (slot->flags & KVM_MEM_READONLY)
-		slot->flags |= KVM_MEMSLOT_INCOHERENT;
+		slot->flags |= KVM_MEM_UNCACHED;
 	return 0;
 }
 
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 8e38878c87c61..5553d112e405b 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -38,6 +38,7 @@
 #define __KVM_HAVE_GUEST_DEBUG
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_UNCACHED_MEM
 
 #define KVM_REG_SIZE(id)						\
 	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 3b934cc94cc83..9dfb519c51e5b 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -43,7 +43,6 @@
  * include/linux/kvm_h.
  */
 #define KVM_MEMSLOT_INVALID	(1UL << 16)
-#define KVM_MEMSLOT_INCOHERENT	(1UL << 17)
 
 /* Two fragments for cross MMIO pages. */
 #define KVM_MAX_MMIO_FRAGMENTS	2
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index a37fd1224f363..4781fc00cd746 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {
  */
 #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
 #define KVM_MEM_READONLY	(1UL << 1)
+#define KVM_MEM_UNCACHED	(1UL << 2)
 
 /* for KVM_IRQ_LINE */
 struct kvm_irq_level {
@@ -753,6 +754,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_PPC_FIXUP_HCALL 103
 #define KVM_CAP_PPC_ENABLE_HCALL 104
 #define KVM_CAP_CHECK_EXTENSION_VM 105
+#define KVM_CAP_UNCACHED_MEM 107
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 59bc56e577b2d..6d63ceb6324d9 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -723,6 +723,10 @@ static int check_memory_region_flags(struct kvm_userspace_memory_region *mem)
 	valid_flags |= KVM_MEM_READONLY;
 #endif
 
+#ifdef __KVM_HAVE_UNCACHED_MEM
+	valid_flags |= KVM_MEM_UNCACHED;
+#endif
+
 	if (mem->flags & ~valid_flags)
 		return -EINVAL;
 
@@ -820,7 +824,8 @@ int __kvm_set_memory_region(struct kvm *kvm,
 		else { /* Modify an existing slot. */
 			if ((mem->userspace_addr != old.userspace_addr) ||
 			    (npages != old.npages) ||
-			    ((new.flags ^ old.flags) & KVM_MEM_READONLY))
+			    ((new.flags ^ old.flags) &
+			     (KVM_MEM_READONLY | KVM_MEM_UNCACHED)))
 				goto out;
 
 			if (base_gfn != old.base_gfn)
-- 
1.8.3.1

Re: [Qemu-devel] [RFC PATCH 1/3] KVM: promote KVM_MEMSLOT_INCOHERENT to uapi

[Christoffer Dall]

On Wed, Mar 18, 2015 at 03:10:31PM -0400, Andrew Jones wrote:
> Also rename to KVM_MEM_UNCACHED.
> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
>  Documentation/virtual/kvm/api.txt | 16 ++++++++++------
>  arch/arm/include/uapi/asm/kvm.h   |  1 +
>  arch/arm/kvm/arm.c                |  1 +
>  arch/arm/kvm/mmu.c                |  4 ++--
>  arch/arm64/include/uapi/asm/kvm.h |  1 +
>  include/linux/kvm_host.h          |  1 -
>  include/uapi/linux/kvm.h          |  2 ++
>  virt/kvm/kvm_main.c               |  7 ++++++-
>  8 files changed, 23 insertions(+), 10 deletions(-)
> 
> diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
> index 0007fef4ed814..a5a51403a7937 100644
> --- a/Documentation/virtual/kvm/api.txt
> +++ b/Documentation/virtual/kvm/api.txt
> @@ -902,6 +902,7 @@ struct kvm_userspace_memory_region {
>  /* for kvm_memory_region::flags */
>  #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
>  #define KVM_MEM_READONLY	(1UL << 1)
> +#define KVM_MEM_UNCACHED	(1UL << 2)
>  
>  This ioctl allows the user to create or modify a guest physical memory
>  slot.  When changing an existing slot, it may be moved in the guest
> @@ -917,12 +918,15 @@ It is recommended that the lower 21 bits of guest_phys_addr and userspace_addr
>  be identical.  This allows large pages in the guest to be backed by large
>  pages in the host.
>  
> -The flags field supports two flags: KVM_MEM_LOG_DIRTY_PAGES and
> -KVM_MEM_READONLY.  The former can be set to instruct KVM to keep track of
> -writes to memory within the slot.  See KVM_GET_DIRTY_LOG ioctl to know how to
> -use it.  The latter can be set, if KVM_CAP_READONLY_MEM capability allows it,
> -to make a new slot read-only.  In this case, writes to this memory will be
> -posted to userspace as KVM_EXIT_MMIO exits.
> +The flags field supports three flags: KVM_MEM_LOG_DIRTY_PAGES,
> +KVM_MEM_READONLY, and KVM_MEM_UNCACHED.  The first can be set to instruct
> +KVM to keep track of writes to memory within the slot.  See KVM_GET_DIRTY_LOG
> +ioctl to know how to use it.  The second can be set, if KVM_CAP_READONLY_MEM
> +capability allows it, to make a new slot read-only.  In this case, writes to
> +this memory will be posted to userspace as KVM_EXIT_MMIO exits. The third can
> +be set, if the KVM_CAP_UNCACHED_MEM capability allows it. This remaps the
> +memory as uncached, i.e. userspace will always directly read/write RAM for
> +this memory region.

I would repeat the definition of the flag in the text here instead of
referring to the 'first', 'second', and 'third' here; it's too hard to
follow.

Also, which combination of these flags are allowed?  Can they all be set
in combination or are they mutually exclusive or a bit of everything?

Thanks,
-Christoffer

[Qemu-devel] [RFC PATCH 2/3] arm/arm64: KVM: decouple READONLY and UNCACHED

[Andrew Jones]

KVM_MEM_UNCACHED memory will no longer need caches to be flushed
for memory as it's faulted in. Just use READONLY directly, in that
case, now.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/kvm/mmu.c | 19 +++++++++----------
 1 file changed, 9 insertions(+), 10 deletions(-)

diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 719931e83c468..59af5ad779eb6 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1260,7 +1260,15 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	if (!hugetlb && !force_pte)
 		hugetlb = transparent_hugepage_adjust(&pfn, &fault_ipa);
 
-	fault_ipa_uncached = memslot->flags & KVM_MEM_UNCACHED;
+	/*
+	 * Readonly memslots are not incoherent with the caches by definition,
+	 * but in practice, they are used mostly to emulate ROMs or NOR flashes
+	 * that the guest may consider devices and hence map as uncached.
+	 * To prevent incoherency issues in these cases, force dcache flushes
+	 * for all pages in the region as they're faulted in.
+	 */
+	fault_ipa_uncached = (memslot->flags & KVM_MEM_READONLY) &&
+				!(memslot->flags & KVM_MEM_UNCACHED);
 
 	if (hugetlb) {
 		pmd_t new_pmd = pfn_pmd(pfn, mem_type);
@@ -1784,15 +1792,6 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 			    unsigned long npages)
 {
-	/*
-	 * Readonly memslots are not incoherent with the caches by definition,
-	 * but in practice, they are used mostly to emulate ROMs or NOR flashes
-	 * that the guest may consider devices and hence map as uncached.
-	 * To prevent incoherency issues in these cases, tag all readonly
-	 * regions as incoherent.
-	 */
-	if (slot->flags & KVM_MEM_READONLY)
-		slot->flags |= KVM_MEM_UNCACHED;
 	return 0;
 }
 
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 3/3] arm/arm64: KVM: implement KVM_MEM_UNCACHED

[Andrew Jones]

When userspace tells us a memory region is uncached, then we
need to pin all its pages and set them all to be uncached.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/include/asm/kvm_mmu.h    |  9 +++++
 arch/arm/include/uapi/asm/kvm.h   |  1 +
 arch/arm/kvm/mmu.c                | 71 +++++++++++++++++++++++++++++++++++++++
 arch/arm64/include/asm/kvm_mmu.h  |  9 +++++
 arch/arm64/include/uapi/asm/kvm.h |  1 +
 5 files changed, 91 insertions(+)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 37ca2a4c6f094..6802f6adc12bf 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -265,6 +265,15 @@ static inline void __kvm_flush_dcache_pud(pud_t pud)
 {
 }
 
+static inline void __set_page_uncached(pte_t *ptep)
+{
+	pte_t pte = *ptep;
+
+	pte = clear_pte_bit(pte, L_PTE_MT_MASK);
+	pte = set_pte_bit(pte, L_PTE_MT_UNCACHED);
+	set_pte_ext(ptep, pte, 0);
+}
+
 #define kvm_virt_to_phys(x)		virt_to_idmap((unsigned long)(x))
 
 void kvm_set_way_flush(struct kvm_vcpu *vcpu);
diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 9d6fc19acf8a2..cdd456f591882 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -109,6 +109,7 @@ struct kvm_sync_regs {
 };
 
 struct kvm_arch_memory_slot {
+	struct page **pages;
 };
 
 /* If you need to interpret the index values, here is the key: */
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 59af5ad779eb6..d4e47572d3a5d 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1697,6 +1697,54 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 		kvm_mmu_wp_memory_region(kvm, mem->slot);
 }
 
+static int set_page_uncached(pte_t *ptep, pgtable_t token,
+			     unsigned long addr, void *data)
+{
+	__set_page_uncached(ptep);
+	kvm_flush_dcache_pte(*ptep);
+	return 0;
+}
+
+static int vma_range_pin_and_set_uncached(struct vm_area_struct *vma,
+					  hva_t start, long nr_pages,
+					  struct page **pages)
+{
+	unsigned long size = nr_pages * PAGE_SIZE;
+	int ret;
+
+	down_read(&vma->vm_mm->mmap_sem);
+	ret = get_user_pages(NULL, vma->vm_mm, start, nr_pages,
+			     true, true, pages, NULL);
+	up_read(&vma->vm_mm->mmap_sem);
+
+	if (ret < 0)
+		return ret;
+
+	if (ret == nr_pages) {
+		ret = apply_to_page_range(vma->vm_mm, start, size,
+					  set_page_uncached, NULL);
+		flush_tlb_kernel_range(start, start + size);
+		return ret;
+	}
+
+	return -EFAULT;
+}
+
+static void unpin_pages(struct kvm_memory_slot *memslot)
+{
+	int i;
+
+	if (!memslot->arch.pages)
+		return;
+
+	for (i = 0; i < memslot->npages; ++i) {
+		if (memslot->arch.pages[i])
+			put_page(memslot->arch.pages[i]);
+	}
+	kfree(memslot->arch.pages);
+	memslot->arch.pages = NULL;
+}
+
 int kvm_arch_prepare_memory_region(struct kvm *kvm,
 				   struct kvm_memory_slot *memslot,
 				   struct kvm_userspace_memory_region *mem,
@@ -1705,6 +1753,7 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
 	hva_t hva = mem->userspace_addr;
 	hva_t reg_end = hva + mem->memory_size;
 	bool writable = !(mem->flags & KVM_MEM_READONLY);
+	struct page **pages = memslot->arch.pages;
 	int ret = 0;
 
 	if (change != KVM_MR_CREATE && change != KVM_MR_MOVE &&
@@ -1768,6 +1817,26 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
 						    writable);
 			if (ret)
 				break;
+		} else if ((change != KVM_MR_FLAGS_ONLY)
+				&& (memslot->flags & KVM_MEM_UNCACHED)) {
+
+			long nr_pages = (vm_end - vm_start)/PAGE_SIZE;
+
+			if (!pages) {
+				pages = kzalloc(memslot->npages *
+					sizeof(struct page *), GFP_KERNEL);
+				if (!pages)
+					return -ENOMEM;
+				memslot->arch.pages = pages;
+			}
+
+			ret = vma_range_pin_and_set_uncached(vma, vm_start,
+							nr_pages, pages);
+			if (ret) {
+				unpin_pages(memslot);
+				break;
+			}
+			pages += nr_pages;
 		}
 		hva = vm_end;
 	} while (hva < reg_end);
@@ -1787,6 +1856,8 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm,
 void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 			   struct kvm_memory_slot *dont)
 {
+	if (free->flags & KVM_MEM_UNCACHED)
+		unpin_pages(free);
 }
 
 int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6458b53731421..f4c3c56587a9f 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -300,6 +300,15 @@ static inline void __kvm_flush_dcache_pud(pud_t pud)
 	kvm_flush_dcache_to_poc(page_address(page), PUD_SIZE);
 }
 
+static inline void __set_page_uncached(pte_t *ptep)
+{
+	pte_t pte = *ptep;
+
+	pte = clear_pte_bit(pte, PTE_ATTRINDX_MASK);
+	pte = set_pte_bit(pte, PTE_ATTRINDX(MT_DEVICE_nGnRnE));
+	set_pte(ptep, pte);
+}
+
 #define kvm_virt_to_phys(x)		__virt_to_phys((unsigned long)(x))
 
 void kvm_set_way_flush(struct kvm_vcpu *vcpu);
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 5553d112e405b..22b4e4a6da950 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -104,6 +104,7 @@ struct kvm_sync_regs {
 };
 
 struct kvm_arch_memory_slot {
+	struct page **pages;
 };
 
 /* If you need to interpret the index values, here is the key: */
-- 
1.8.3.1

Re: [Qemu-devel] [RFC PATCH 0/3] KVM: Introduce KVM_MEM_UNCACHED

[Paolo Bonzini]

On 18/03/2015 20:10, Andrew Jones wrote:
> Introduce a new memory region flag, KVM_MEM_UNCACHED, which
> is needed by ARM. This flag informs KVM that the given memory
> region is typically mapped by the guest as uncached. KVM for
> ARM then maps that region as uncached for userspace as well,
> in order to keep coherency.
> 
> Andrew Jones (3):
>   KVM: promote KVM_MEMSLOT_INCOHERENT to uapi
>   arm/arm64: KVM: decouple READONLY and UNCACHED
>   arm/arm64: KVM: implement KVM_MEM_UNCACHED
> 
>  Documentation/virtual/kvm/api.txt | 16 ++++---
>  arch/arm/include/asm/kvm_mmu.h    |  9 ++++
>  arch/arm/include/uapi/asm/kvm.h   |  2 +
>  arch/arm/kvm/arm.c                |  1 +
>  arch/arm/kvm/mmu.c                | 90 ++++++++++++++++++++++++++++++++++-----
>  arch/arm64/include/asm/kvm_mmu.h  |  9 ++++
>  arch/arm64/include/uapi/asm/kvm.h |  2 +
>  include/linux/kvm_host.h          |  1 -
>  include/uapi/linux/kvm.h          |  2 +
>  virt/kvm/kvm_main.c               |  7 ++-
>  10 files changed, 121 insertions(+), 18 deletions(-)
> 

I think the pinning breaks running KVM as non-root (the default ulimit
-l is just 64 KiB).  Can you do something about it with the MMU
notifiers instead?

It looks very clean.

The fly in the ointment is that some kind of quirk is probably needed in
the future for ivshmem---it's definitely not acceptable performance-wise
to mark it uncached, neither in the guests not in the host.  On the
other hand that quirk would be necessary only in the guest, not in the
firmware, so we can live with a mixture of the two approaches.

Paolo

Re: [Qemu-devel] [RFC PATCH 0/3] KVM: Introduce KVM_MEM_UNCACHED

[Andrew Jones]

On Thu, Mar 19, 2015 at 05:56:20PM +0100, Paolo Bonzini wrote:
> 
> 
> On 18/03/2015 20:10, Andrew Jones wrote:
> > Introduce a new memory region flag, KVM_MEM_UNCACHED, which
> > is needed by ARM. This flag informs KVM that the given memory
> > region is typically mapped by the guest as uncached. KVM for
> > ARM then maps that region as uncached for userspace as well,
> > in order to keep coherency.
> > 
> > Andrew Jones (3):
> >   KVM: promote KVM_MEMSLOT_INCOHERENT to uapi
> >   arm/arm64: KVM: decouple READONLY and UNCACHED
> >   arm/arm64: KVM: implement KVM_MEM_UNCACHED
> > 
> >  Documentation/virtual/kvm/api.txt | 16 ++++---
> >  arch/arm/include/asm/kvm_mmu.h    |  9 ++++
> >  arch/arm/include/uapi/asm/kvm.h   |  2 +
> >  arch/arm/kvm/arm.c                |  1 +
> >  arch/arm/kvm/mmu.c                | 90 ++++++++++++++++++++++++++++++++++-----
> >  arch/arm64/include/asm/kvm_mmu.h  |  9 ++++
> >  arch/arm64/include/uapi/asm/kvm.h |  2 +
> >  include/linux/kvm_host.h          |  1 -
> >  include/uapi/linux/kvm.h          |  2 +
> >  virt/kvm/kvm_main.c               |  7 ++-
> >  10 files changed, 121 insertions(+), 18 deletions(-)
> > 

Hi Paolo,

Thanks for the comments!

> 
> I think the pinning breaks running KVM as non-root (the default ulimit
> -l is just 64 KiB).  Can you do something about it with the MMU
> notifiers instead?

I'll look into this as soon as possible. I'm headed out 25 minutes ago
for some vacation time though, so it'll be week or so before I can.

> 
> It looks very clean.
> 
> The fly in the ointment is that some kind of quirk is probably needed in
> the future for ivshmem---it's definitely not acceptable performance-wise
> to mark it uncached, neither in the guests not in the host.  On the
> other hand that quirk would be necessary only in the guest, not in the
> firmware, so we can live with a mixture of the two approaches.

Yes, quirking ivshmem makes sense. ivshmem users (guest drivers) should
ensure they map the memory as cached, as they know it's normal memory
on the host.

drew

Re: [Qemu-devel] [RFC PATCH 0/3] KVM: Introduce KVM_MEM_UNCACHED

[Alexander Graf]

On 18.03.15 20:10, Andrew Jones wrote:
> Introduce a new memory region flag, KVM_MEM_UNCACHED, which
> is needed by ARM. This flag informs KVM that the given memory
> region is typically mapped by the guest as uncached. KVM for
> ARM then maps that region as uncached for userspace as well,
> in order to keep coherency.

I finally managed to give this a spin and immediately ran into an
unaligned access trap:

[  116.509976] Unhandled fault: alignment fault (0x92000061) at
0x000003ffb1300000

Program received signal SIGBUS, Bus error.
[Switching to Thread 0x3ffb317ecb0 (LWP 1956)]
0x000003ffb685ed68 in memset () from /lib64/libc.so.6
(gdb) bt
#0  0x000003ffb685ed68 in memset () from /lib64/libc.so.6
#1  0x000002aaaac013ec in memset (__len=<optimized out>, __ch=0,
__dest=<optimized out>) at /usr/include/bits/string3.h:90
#2  vbe_ioport_write_data (opaque=0x2aaabbd3600, addr=<optimized out>,
val=65) at /usr/src/debug/qemu-2.3.0-rc4/hw/display/vga.c:739
#3  0x000002aaaabe07dc in memory_region_write_accessor (mr=<optimized
out>, addr=<optimized out>, value=<optimized out>, size=<optimized out>,
    shift=<optimized out>, mask=<optimized out>) at
/usr/src/debug/qemu-2.3.0-rc4/memory.c:430
[...]

So maybe forcing the guest to see VRAM as cached is the safer choice
after all?

Alex

Re: [Qemu-devel] [RFC PATCH 0/3] KVM: Introduce KVM_MEM_UNCACHED

[Peter Maydell]

On 29 April 2015 at 10:03, Alexander Graf <agraf@suse.de> wrote:
>
>
> On 18.03.15 20:10, Andrew Jones wrote:
>> Introduce a new memory region flag, KVM_MEM_UNCACHED, which
>> is needed by ARM. This flag informs KVM that the given memory
>> region is typically mapped by the guest as uncached. KVM for
>> ARM then maps that region as uncached for userspace as well,
>> in order to keep coherency.
>
> I finally managed to give this a spin and immediately ran into an
> unaligned access trap:
>
> [  116.509976] Unhandled fault: alignment fault (0x92000061) at
> 0x000003ffb1300000
>
> Program received signal SIGBUS, Bus error.
> [Switching to Thread 0x3ffb317ecb0 (LWP 1956)]
> 0x000003ffb685ed68 in memset () from /lib64/libc.so.6
> (gdb) bt
> #0  0x000003ffb685ed68 in memset () from /lib64/libc.so.6
> #1  0x000002aaaac013ec in memset (__len=<optimized out>, __ch=0,
> __dest=<optimized out>) at /usr/include/bits/string3.h:90
> #2  vbe_ioport_write_data (opaque=0x2aaabbd3600, addr=<optimized out>,
> val=65) at /usr/src/debug/qemu-2.3.0-rc4/hw/display/vga.c:739
> #3  0x000002aaaabe07dc in memory_region_write_accessor (mr=<optimized
> out>, addr=<optimized out>, value=<optimized out>, size=<optimized out>,
>     shift=<optimized out>, mask=<optimized out>) at
> /usr/src/debug/qemu-2.3.0-rc4/memory.c:430
> [...]

This appears to be because the KVM_MEM_UNCACHED flag
confusingly isn't marking the memory as Normal-Noncacheable
but as Device-nGnRnE (aka Strongly Ordered). You can't unalignedly
access Device memory (and so you can't use the usual userspace
memcpy, memset, etc).

Did we really want Device-nGnRnE memory here? If we did can
we have a less confusing name for the flag?

[for the non-ARM folk, nGnRnE == no gathering of accesses,
no reordering, no early-write-acknowledgement.]

thanks
-- PMM

Re: [Qemu-devel] [RFC PATCH 0/3] KVM: Introduce KVM_MEM_UNCACHED

[Andrew Jones]

On Wed, Apr 29, 2015 at 10:19:55AM +0100, Peter Maydell wrote:
> On 29 April 2015 at 10:03, Alexander Graf <agraf@suse.de> wrote:
> >
> >
> > On 18.03.15 20:10, Andrew Jones wrote:
> >> Introduce a new memory region flag, KVM_MEM_UNCACHED, which
> >> is needed by ARM. This flag informs KVM that the given memory
> >> region is typically mapped by the guest as uncached. KVM for
> >> ARM then maps that region as uncached for userspace as well,
> >> in order to keep coherency.
> >
> > I finally managed to give this a spin and immediately ran into an
> > unaligned access trap:

Thanks! And err.. sorry it broke.

> >
> > [  116.509976] Unhandled fault: alignment fault (0x92000061) at
> > 0x000003ffb1300000
> >
> > Program received signal SIGBUS, Bus error.
> > [Switching to Thread 0x3ffb317ecb0 (LWP 1956)]
> > 0x000003ffb685ed68 in memset () from /lib64/libc.so.6
> > (gdb) bt
> > #0  0x000003ffb685ed68 in memset () from /lib64/libc.so.6
> > #1  0x000002aaaac013ec in memset (__len=<optimized out>, __ch=0,
> > __dest=<optimized out>) at /usr/include/bits/string3.h:90
> > #2  vbe_ioport_write_data (opaque=0x2aaabbd3600, addr=<optimized out>,
> > val=65) at /usr/src/debug/qemu-2.3.0-rc4/hw/display/vga.c:739
> > #3  0x000002aaaabe07dc in memory_region_write_accessor (mr=<optimized
> > out>, addr=<optimized out>, value=<optimized out>, size=<optimized out>,
> >     shift=<optimized out>, mask=<optimized out>) at
> > /usr/src/debug/qemu-2.3.0-rc4/memory.c:430
> > [...]
> 
> This appears to be because the KVM_MEM_UNCACHED flag
> confusingly isn't marking the memory as Normal-Noncacheable
> but as Device-nGnRnE (aka Strongly Ordered). You can't unalignedly
> access Device memory (and so you can't use the usual userspace
> memcpy, memset, etc).
> 
> Did we really want Device-nGnRnE memory here? If we did can
> we have a less confusing name for the flag?

I hadn't considered the other side-effects of going with device
vs. normal memory. I'll change patch 3/3 with

31c31
< +	pte = set_pte_bit(pte, L_PTE_MT_UNCACHED);
---
> +	pte = set_pte_bit(pte, L_PTE_MT_WRITEBACK);
166c166
< +	pte = set_pte_bit(pte, PTE_ATTRINDX(MT_DEVICE_nGnRnE));
---
> +	pte = set_pte_bit(pte, PTE_ATTRINDX(MT_NORMAL_NC));



I've left this on the backburner too long, but I can pick it back
up on Monday. I still haven't looked at Paolo's mmu notifier
suggestion yet either. I'll start looking at that as well.

Thanks,
drew


> 
> [for the non-ARM folk, nGnRnE == no gathering of accesses,
> no reordering, no early-write-acknowledgement.]
> 
> thanks
> -- PMM
> 
> 

[Qemu-devel] [RFC PATCH 0/4] support KVM_MEM_UNCACHED

[Andrew Jones]

Add support for the new KVM_MEM_UNCACHED flag, and flag
appropriate memory. (Only flags vga/vram for now.)

Andrew Jones (4):
  kvm-all: put kvm_mem_flags to more work
  HACK: linux header update
  memory: add uncached flag
  vga: flag vram as uncached

 hw/display/vga.c          |  1 +
 include/exec/memory.h     | 25 +++++++++++++++++++++++++
 kvm-all.c                 | 34 ++++++++++++++++++++++++----------
 linux-headers/linux/kvm.h |  2 ++
 memory.c                  | 15 +++++++++++++++
 5 files changed, 67 insertions(+), 10 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 1/4] kvm-all: put kvm_mem_flags to more work

[Andrew Jones]

Currently kvm_mem_flags just translates bools to bits, let's
make it also determine the bools first. This avoids its parameter
list growing each time we add a flag.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 kvm-all.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 55025cc366992..a1bc05a8d5c5c 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -295,10 +295,14 @@ err:
  * dirty pages logging control
  */
 
-static int kvm_mem_flags(KVMState *s, bool log_dirty, bool readonly)
+static int kvm_mem_flags(MemoryRegion *mr)
 {
+    bool readonly = mr->readonly || memory_region_is_romd(mr);
     int flags = 0;
-    flags = log_dirty ? KVM_MEM_LOG_DIRTY_PAGES : 0;
+
+    if (memory_region_is_logging(mr)) {
+        flags |= KVM_MEM_LOG_DIRTY_PAGES;
+    }
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
@@ -313,7 +317,10 @@ static int kvm_slot_dirty_pages_log_change(KVMSlot *mem, bool log_dirty)
 
     old_flags = mem->flags;
 
-    flags = (mem->flags & ~mask) | kvm_mem_flags(s, log_dirty, false);
+    flags = mem->flags & ~mask;
+    if (log_dirty) {
+        flags |= KVM_MEM_LOG_DIRTY_PAGES;
+    }
     mem->flags = flags;
 
     /* If nothing changed effectively, no need to issue ioctl */
@@ -643,9 +650,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
     KVMSlot *mem, old;
     int err;
     MemoryRegion *mr = section->mr;
-    bool log_dirty = memory_region_is_logging(mr);
     bool writeable = !mr->readonly && !mr->rom_device;
-    bool readonly_flag = mr->readonly || memory_region_is_romd(mr);
     hwaddr start_addr = section->offset_within_address_space;
     ram_addr_t size = int128_get64(section->size);
     void *ram = NULL;
@@ -689,7 +694,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             (ram - start_addr == mem->ram - mem->start_addr)) {
             /* The new slot fits into the existing one and comes with
              * identical parameters - update flags and done. */
-            kvm_slot_dirty_pages_log_change(mem, log_dirty);
+            kvm_slot_dirty_pages_log_change(mem, memory_region_is_logging(mr));
             return;
         }
 
@@ -722,7 +727,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             mem->memory_size = old.memory_size;
             mem->start_addr = old.start_addr;
             mem->ram = old.ram;
-            mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags = kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -743,7 +748,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             mem->memory_size = start_addr - old.start_addr;
             mem->start_addr = old.start_addr;
             mem->ram = old.ram;
-            mem->flags =  kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags =  kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -767,7 +772,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             size_delta = mem->start_addr - old.start_addr;
             mem->memory_size = old.memory_size - size_delta;
             mem->ram = old.ram + size_delta;
-            mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags = kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -789,7 +794,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
     mem->memory_size = size;
     mem->start_addr = start_addr;
     mem->ram = ram;
-    mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+    mem->flags = kvm_mem_flags(mr);
 
     err = kvm_set_user_memory_region(s, mem);
     if (err) {
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 2/4] HACK: linux header update

[Andrew Jones]

Should do a proper update-linux-headers.sh update.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 linux-headers/linux/kvm.h | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 60a54c82a3b76..34c03cd80aa69 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {
  */
 #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
 #define KVM_MEM_READONLY	(1UL << 1)
+#define KVM_MEM_UNCACHED	(1UL << 2)
 
 /* for KVM_IRQ_LINE */
 struct kvm_irq_level {
@@ -760,6 +761,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_PPC_ENABLE_HCALL 104
 #define KVM_CAP_CHECK_EXTENSION_VM 105
 #define KVM_CAP_S390_USER_SIGP 106
+#define KVM_CAP_UNCACHED_MEM 107
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 3/4] memory: add uncached flag

[Andrew Jones]

Add an 'uncached' flag, which will result in the KVM_MEM_UNCACHED
flag getting set on KVM's corresponding memory slot.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 include/exec/memory.h | 25 +++++++++++++++++++++++++
 kvm-all.c             |  9 +++++++++
 memory.c              | 15 +++++++++++++++
 3 files changed, 49 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 06ffa1d185b93..2a0d016f5fe6d 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -158,6 +158,7 @@ struct MemoryRegion {
     bool rom_device;
     bool warning_printed; /* For reservations */
     bool flush_coalesced_mmio;
+    bool uncached;
     MemoryRegion *alias;
     hwaddr alias_offset;
     int32_t priority;
@@ -778,6 +779,30 @@ void memory_region_set_flush_coalesced(MemoryRegion *mr);
 void memory_region_clear_flush_coalesced(MemoryRegion *mr);
 
 /**
+ * memory_region_set_uncached: Flag this memory region (for e.g. kvm) as
+ *                             typically being mapped uncached.
+ *
+ * @mr: the memory region to be updated.
+ */
+void memory_region_set_uncached(MemoryRegion *mr);
+
+/**
+ * memory_region_clear_uncached: Remove the 'uncached' flag.
+ *
+ * @mr: the memory region to be updated.
+ */
+void memory_region_clear_uncached(MemoryRegion *mr);
+
+/**
+ * memory_region_may_map_uncached: Return the 'uncached' flag, which
+ *                                 indicates the region is typically
+ *                                 mapped uncached.
+ *
+ * @mr: the memory region to check.
+ */
+bool memory_region_may_map_uncached(MemoryRegion *mr);
+
+/**
  * memory_region_add_eventfd: Request an eventfd to be triggered when a word
  *                            is written to a location.
  *
diff --git a/kvm-all.c b/kvm-all.c
index a1bc05a8d5c5c..4c826d0eab37b 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -126,6 +126,7 @@ bool kvm_gsi_routing_allowed;
 bool kvm_gsi_direct_mapping;
 bool kvm_allowed;
 bool kvm_readonly_mem_allowed;
+bool kvm_uncached_mem_allowed;
 bool kvm_vm_attributes_allowed;
 
 static const KVMCapabilityInfo kvm_required_capabilites[] = {
@@ -306,6 +307,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
+    if (memory_region_may_map_uncached(mr) && kvm_uncached_mem_allowed) {
+        flags |= KVM_MEM_UNCACHED;
+    }
     return flags;
 }
 
@@ -1595,6 +1599,11 @@ static int kvm_init(MachineState *ms)
         (kvm_check_extension(s, KVM_CAP_READONLY_MEM) > 0);
 #endif
 
+#ifdef KVM_CAP_UNCACHED_MEM
+    kvm_uncached_mem_allowed =
+        (kvm_check_extension(s, KVM_CAP_UNCACHED_MEM) > 0);
+#endif
+
     kvm_eventfds_allowed =
         (kvm_check_extension(s, KVM_CAP_IOEVENTFD) > 0);
 
diff --git a/memory.c b/memory.c
index ee3f2a8a954ef..495cec1b6650c 100644
--- a/memory.c
+++ b/memory.c
@@ -1549,6 +1549,21 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr)
     }
 }
 
+void memory_region_set_uncached(MemoryRegion *mr)
+{
+    mr->uncached = true;
+}
+
+void memory_region_clear_uncached(MemoryRegion *mr)
+{
+    mr->uncached = false;
+}
+
+bool memory_region_may_map_uncached(MemoryRegion *mr)
+{
+    return mr->uncached;
+}
+
 void memory_region_add_eventfd(MemoryRegion *mr,
                                hwaddr addr,
                                unsigned size,
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 4/4] vga: flag vram as uncached

[Andrew Jones]

Video RAM is typically mapped as uncached by guests. Flag it as
such.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 hw/display/vga.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/display/vga.c b/hw/display/vga.c
index c0f7b343bbab5..5c8c249a8d780 100644
--- a/hw/display/vga.c
+++ b/hw/display/vga.c
@@ -2143,6 +2143,7 @@ void vga_common_init(VGACommonState *s, Object *obj, bool global_vmstate)
     s->is_vbe_vmstate = 1;
     memory_region_init_ram(&s->vram, obj, "vga.vram", s->vram_size,
                            &error_abort);
+    memory_region_set_uncached(&s->vram);
     vmstate_register_ram(&s->vram, global_vmstate ? NULL : DEVICE(obj));
     xen_register_framebuffer(&s->vram);
     s->vram_ptr = memory_region_get_ram_ptr(&s->vram);
-- 
1.8.3.1

Re: [Qemu-devel] the arm cache coherency cluster "v2"

[Andrew Jones]

On Wed, Mar 18, 2015 at 03:08:20PM -0400, Andrew Jones wrote:
> In reply to this message I'll send two series' one for KVM and
> one for QEMU. The two series' are their respective component
> complements, and attempt to implement cache coherency for arm
> guests using emulated devices, where the emulator (qemu) uses
> cached memory for the device memory, but the guest uses
> uncached - as device memory is generally used. Right now I've
> just focused on VGA vram.
> 
> This approach is the "MADV_UNCACHED" type that Paolo suggested.
> This type of approach could also be described as "make userspace's
> memory access type match the expected access type of the guest",
> and Mario has suggested using a memory driver, which could have
> the same result.
> 
> The coming series' is inspired by both Paolo's and Mario's
> suggestions, but it uses a kvm memslot flag, rather than an
> madvise flag, and thus for the memory driver, it's just KVM.
> 
> See the thread
> 
> https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg01254.html
> 
> for some more background.
> 
> Thanks in advance for comments.
> 
> drew

I forgot to mention that I've done some light testing with this.
It seems to work, and without (to eye) noticeable performance
degradation.

Thanks,
drew

Re: [Qemu-devel] the arm cache coherency cluster "v2"

[Alexander Graf]

On 18.03.15 20:18, Andrew Jones wrote:
> On Wed, Mar 18, 2015 at 03:08:20PM -0400, Andrew Jones wrote:
>> In reply to this message I'll send two series' one for KVM and
>> one for QEMU. The two series' are their respective component
>> complements, and attempt to implement cache coherency for arm
>> guests using emulated devices, where the emulator (qemu) uses
>> cached memory for the device memory, but the guest uses
>> uncached - as device memory is generally used. Right now I've
>> just focused on VGA vram.
>>
>> This approach is the "MADV_UNCACHED" type that Paolo suggested.
>> This type of approach could also be described as "make userspace's
>> memory access type match the expected access type of the guest",
>> and Mario has suggested using a memory driver, which could have
>> the same result.
>>
>> The coming series' is inspired by both Paolo's and Mario's
>> suggestions, but it uses a kvm memslot flag, rather than an
>> madvise flag, and thus for the memory driver, it's just KVM.
>>
>> See the thread
>>
>> https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg01254.html
>>
>> for some more background.
>>
>> Thanks in advance for comments.
>>
>> drew
> 
> I forgot to mention that I've done some light testing with this.
> It seems to work, and without (to eye) noticeable performance
> degradation.

Just for the record, I couldn't get it to work :). But I'm looking
forward to the next version with MMU notifiers!


Alex