[PATCH v2 0/3] arm/arm64: Add dirty page logging for ARMv8 (3.17.0-rc1)

[PATCH v2 0/3] arm/arm64: Add dirty page logging for ARMv8 (3.17.0-rc1)

[m.smarduch at samsung.com]

From: Mario Smarduch <m.smarduch@samsung.com>

This patch series adds support for ARMv8 dirty page logging. It applies on top
of ARMv7 patch series. This feature hooks into generic dirty page logging,
and reuses earlier ARMv7 architecture support added with few ARMv8 additions.

Testing:
--------
- ARMv8 - Tested on Foundation Model.
- ARMv7 - retested on Exynos 5440 development board.

Test Description:
- Added mmio device to QEMU 'machvirt' with on board memory (1MB in this case),
  Device memory is Posix shared memory segment visible to host. Enabled
  dirty logging for that memslot.
- Added memslot migration thread to export dirty bit map to host.
- Implemented memory migration thread on host.

Operation:
  o On Guest application mmaps() the region (via /dev/mem) and dirties pages.
  o The host migration thread does a pre-copy of /dev/shm/aeshmem to a host
    file, repeatedly requests QEMU for memory region dirty page log,
    incrementally copies dirty pages from /dev/shm/aeshmem to host file.
  o Guest application is stopped and both /dev/shm/aeshmem and host file are
    checksummed to check for match to validate dirty page log applied
    incremental updates, validating logging.

See https://github.com/mjsmar/arm-dirtylog-tests 'v8' subfolder

ARMv7/v8 Dirty page logging implementation overivew-
- initially write protects memory region pages 2nd stage page tables
- add support to read dirty page log and again write protect dirty pages 
  for next pass.
- second stage huge page are dissolved into small page tables to keep track of
  dirty pages at page granularity. Tracking at huge page granularity limits
  migration to an almost idle system. Small page size logging supports higher
  memory dirty rates.
- In the event migration is canceled, normal behavior is resumed huge pages
  are rebuilt over time.

Changes Since (v1):
- Rebased to ARMv7 version
- Addressed Christoffers review comments.

Mario Smarduch (3):
  KVM: arm64: ARMv8 header changes for page logging
  KVM: arm64: Add HYP interface to flush VM Stage 1/2nd TLB entires
  KVM: arm/arm64: Enable Dirty Page logging for ARMv8

 arch/arm/include/asm/kvm_host.h        | 12 ------------
 arch/arm/kvm/arm.c                     |  4 ----
 arch/arm/kvm/mmu.c                     | 20 +++++++++++---------
 arch/arm64/include/asm/kvm_asm.h       |  1 +
 arch/arm64/include/asm/kvm_host.h      |  1 +
 arch/arm64/include/asm/kvm_mmu.h       | 20 ++++++++++++++++++++
 arch/arm64/include/asm/pgtable-hwdef.h |  1 +
 arch/arm64/kvm/Kconfig                 |  2 ++
 arch/arm64/kvm/hyp.S                   | 22 ++++++++++++++++++++++
 9 files changed, 58 insertions(+), 25 deletions(-)

-- 
1.9.1

[PATCH v2 1/3] KVM: arm64: ARMv8 header changes for page logging

[m.smarduch at samsung.com]

From: Mario Smarduch <m.smarduch@samsung.com>

This patch adds arm64 helpers to write protect pmds/ptes and retrieve 
permissions while logging dirty pages. Also adds prototype to write protect 
a memory slot and adds a pmd define to check for read-only pmds.

Signed-off-by: Mario Smarduch <m.smarduch@samsung.com>
---
 arch/arm64/include/asm/kvm_asm.h       |  1 +
 arch/arm64/include/asm/kvm_host.h      |  1 +
 arch/arm64/include/asm/kvm_mmu.h       | 20 ++++++++++++++++++++
 arch/arm64/include/asm/pgtable-hwdef.h |  1 +
 4 files changed, 23 insertions(+)

diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 4838421..4f7310f 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -126,6 +126,7 @@ extern char __kvm_hyp_vector[];
 
 extern void __kvm_flush_vm_context(void);
 extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
+extern void __kvm_tlb_flush_vmid(struct kvm *kvm);
 
 extern int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
 
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index e10c45a..7548dbc 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -196,6 +196,7 @@ struct kvm_vcpu *kvm_arm_get_running_vcpu(void);
 struct kvm_vcpu __percpu **kvm_get_running_vcpus(void);
 
 u64 kvm_call_hyp(void *hypfn, ...);
+void kvm_mmu_wp_memory_region(struct kvm *kvm, int slot);
 
 int handle_exit(struct kvm_vcpu *vcpu, struct kvm_run *run,
 		int exception_index);
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 8e138c7..8b0ffe3 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -121,6 +121,26 @@ static inline void kvm_set_s2pmd_writable(pmd_t *pmd)
 	pmd_val(*pmd) |= PMD_S2_RDWR;
 }
 
+static inline void kvm_set_s2pte_readonly(pte_t *pte)
+{
+	pte_val(*pte) = (pte_val(*pte) & ~PTE_S2_RDWR) | PTE_S2_RDONLY;
+}
+
+static inline bool kvm_s2pte_readonly(pte_t *pte)
+{
+	return (pte_val(*pte) & PTE_S2_RDWR) == PTE_S2_RDONLY;
+}
+
+static inline void kvm_set_s2pmd_readonly(pmd_t *pmd)
+{
+	pmd_val(*pmd) = (pmd_val(*pmd) & ~PMD_S2_RDWR) | PMD_S2_RDONLY;
+}
+
+static inline bool kvm_s2pmd_readonly(pmd_t *pmd)
+{
+	return (pmd_val(*pmd) & PMD_S2_RDWR) == PMD_S2_RDONLY;
+}
+
 #define kvm_pgd_addr_end(addr, end)	pgd_addr_end(addr, end)
 #define kvm_pud_addr_end(addr, end)	pud_addr_end(addr, end)
 #define kvm_pmd_addr_end(addr, end)	pmd_addr_end(addr, end)
diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
index 88174e0..5f930cc 100644
--- a/arch/arm64/include/asm/pgtable-hwdef.h
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -119,6 +119,7 @@
 #define PTE_S2_RDONLY		(_AT(pteval_t, 1) << 6)   /* HAP[2:1] */
 #define PTE_S2_RDWR		(_AT(pteval_t, 3) << 6)   /* HAP[2:1] */
 
+#define PMD_S2_RDONLY		(_AT(pmdval_t, 1) << 6)   /* HAP[2:1] */
 #define PMD_S2_RDWR		(_AT(pmdval_t, 3) << 6)   /* HAP[2:1] */
 
 /*
-- 
1.9.1

[PATCH v2 2/3] KVM: arm64: Add HYP interface to flush VM TLB entires

[m.smarduch at samsung.com]

From: Mario Smarduch <m.smarduch@samsung.com>

This patch adds support for arm64 hyp interface to flush all TLBs associated
with VMID.

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Signed-off-by: Mario Smarduch <m.smarduch@samsung.com>
---
 arch/arm64/kvm/hyp.S | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
index b72aa9f..6e1b5df 100644
--- a/arch/arm64/kvm/hyp.S
+++ b/arch/arm64/kvm/hyp.S
@@ -1030,6 +1030,28 @@ ENTRY(__kvm_tlb_flush_vmid_ipa)
 	ret
 ENDPROC(__kvm_tlb_flush_vmid_ipa)
 
+/**
+ * void __kvm_tlb_flush_vmid(struct kvm *kvm) - Flush per-VMID TLBs
+ * @struct kvm *kvm - pointer to kvm structure
+ *
+ * Invalidates all Stage 1 and 2 TLB entries for current VMID.
+ */
+ENTRY(__kvm_tlb_flush_vmid)
+	dsb     ishst
+
+	kern_hyp_va     x0
+	ldr     x2, [x0, #KVM_VTTBR]
+	msr     vttbr_el2, x2
+	isb
+
+	tlbi    vmalls12e1is
+	dsb     ish
+	isb
+
+	msr     vttbr_el2, xzr
+	ret
+ENDPROC(__kvm_tlb_flush_vmid)
+
 ENTRY(__kvm_flush_vm_context)
 	dsb	ishst
 	tlbi	alle1is
-- 
1.9.1

[PATCH v2 3/3] KVM: arm/arm64: Enable Dirty Page logging for ARMv8

[m.smarduch at samsung.com]

From: Mario Smarduch <m.smarduch@samsung.com>

This patch enables ARMv8 ditry page logging support. Plugs ARMv8 into generic
layer through Kconfig symbol, and drops earlier ARM64 constraints to enable
logging at architecture layer.

Signed-off-by: Mario Smarduch <m.smarduch@samsung.com>
---
 arch/arm/include/asm/kvm_host.h | 12 ------------
 arch/arm/kvm/arm.c              |  4 ----
 arch/arm/kvm/mmu.c              | 20 +++++++++++---------
 arch/arm64/kvm/Kconfig          |  2 ++
 4 files changed, 13 insertions(+), 25 deletions(-)

diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 8fa6238..21f828e 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -220,18 +220,6 @@ static inline void __cpu_init_hyp_mode(phys_addr_t boot_pgd_ptr,
 	kvm_call_hyp((void*)hyp_stack_ptr, vector_ptr, pgd_ptr);
 }
 
-/**
- * kvm_flush_remote_tlbs() - flush all VM TLB entries
- * @kvm:	pointer to kvm structure.
- *
- * Interface to HYP function to flush all VM TLB entries without address
- * parameter.
- */
-static inline void kvm_flush_remote_tlbs(struct kvm *kvm)
-{
-	kvm_call_hyp(__kvm_tlb_flush_vmid, kvm);
-}
-
 static inline int kvm_arch_dev_ioctl_check_extension(long ext)
 {
 	return 0;
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index 040c0f3..8d27e94 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -759,7 +759,6 @@ long kvm_arch_vcpu_ioctl(struct file *filp,
  */
 int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)
 {
-#ifdef CONFIG_ARM
 	int r;
 	bool is_dirty = false;
 
@@ -783,9 +782,6 @@ out:
 	mutex_unlock(&kvm->slots_lock);
 
 	return r;
-#else /* ARM64 */
-	return -EINVAL;
-#endif
 }
 
 static int kvm_vm_ioctl_set_device_addr(struct kvm *kvm,
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index d29de77..95df040 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -52,11 +52,18 @@ static phys_addr_t hyp_idmap_vector;
 
 static bool kvm_get_logging_state(struct kvm_memory_slot *memslot)
 {
-#ifdef CONFIG_ARM
 	return !!memslot->dirty_bitmap;
-#else
-	return false;
-#endif
+}
+
+/**
+ * kvm_flush_remote_tlbs() - flush all VM TLB entries for v7/8
+ * @kvm:	pointer to kvm structure.
+ *
+ * Interface to HYP function to flush all VM TLB entries
+ */
+inline void kvm_flush_remote_tlbs(struct kvm *kvm)
+{
+	kvm_call_hyp(__kvm_tlb_flush_vmid, kvm);
 }
 
 static void kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
@@ -775,7 +782,6 @@ static bool transparent_hugepage_adjust(pfn_t *pfnp, phys_addr_t *ipap)
 	return false;
 }
 
-#ifdef CONFIG_ARM
 /**
  * stage2_wp_ptes - write protect PMD range
  * @pmd:	pointer to pmd entry
@@ -920,7 +926,6 @@ void kvm_arch_mmu_write_protect_pt_masked(struct kvm *kvm,
 
 	stage2_wp_range(kvm, start, end);
 }
-#endif
 
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 			  struct kvm_memory_slot *memslot,
@@ -1323,7 +1328,6 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 		spin_unlock(&kvm->mmu_lock);
 	}
 
-#ifdef CONFIG_ARM
 	/*
 	 * At this point memslot has been committed and there is an
 	 * allocated dirty_bitmap[], dirty pages will be be tracked while the
@@ -1332,8 +1336,6 @@ void kvm_arch_commit_memory_region(struct kvm *kvm,
 	if (change != KVM_MR_DELETE && change != KVM_MR_MOVE &&
 					mem->flags & KVM_MEM_LOG_DIRTY_PAGES)
 		kvm_mmu_wp_memory_region(kvm, mem->slot);
-#endif
-
 }
 
 int kvm_arch_prepare_memory_region(struct kvm *kvm,
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 8ba85e9..3ce389b 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -22,10 +22,12 @@ config KVM
 	select PREEMPT_NOTIFIERS
 	select ANON_INODES
 	select HAVE_KVM_CPU_RELAX_INTERCEPT
+	select HAVE_KVM_ARCH_TLB_FLUSH_ALL
 	select KVM_MMIO
 	select KVM_ARM_HOST
 	select KVM_ARM_VGIC
 	select KVM_ARM_TIMER
+	select KVM_GENERIC_DIRTYLOG_READ_PROTECT
 	---help---
 	  Support hosting virtualized guest machines.
 
-- 
1.9.1

[PATCH v2 1/3] KVM: arm64: ARMv8 header changes for page logging

[Christoffer Dall]

On Sat, Nov 15, 2014 at 12:19:08AM -0800, m.smarduch at samsung.com wrote:
> From: Mario Smarduch <m.smarduch@samsung.com>
> 
> This patch adds arm64 helpers to write protect pmds/ptes and retrieve 
> permissions while logging dirty pages. Also adds prototype to write protect 
> a memory slot and adds a pmd define to check for read-only pmds.
> 
> Signed-off-by: Mario Smarduch <m.smarduch@samsung.com>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

[PATCH v2 3/3] KVM: arm/arm64: Enable Dirty Page logging for ARMv8

[Christoffer Dall]

On Sat, Nov 15, 2014 at 12:19:10AM -0800, m.smarduch at samsung.com wrote:
> From: Mario Smarduch <m.smarduch@samsung.com>
> 
> This patch enables ARMv8 ditry page logging support. Plugs ARMv8 into generic
> layer through Kconfig symbol, and drops earlier ARM64 constraints to enable
> logging at architecture layer.
> 
> Signed-off-by: Mario Smarduch <m.smarduch@samsung.com>

Just reminding you again of what I said in the previous thread (think
that was before you sent this out), that you need to handle the pud_huge
case in arch/arm/kvm/mmu.c for ARMv8 here.

-Christoffer

[PATCH v2 3/3] KVM: arm/arm64: Enable Dirty Page logging for ARMv8

[Mario Smarduch]

On 11/22/2014 12:02 PM, Christoffer Dall wrote:
> On Sat, Nov 15, 2014 at 12:19:10AM -0800, m.smarduch at samsung.com wrote:
>> From: Mario Smarduch <m.smarduch@samsung.com>
>>
>> This patch enables ARMv8 ditry page logging support. Plugs ARMv8 into generic
>> layer through Kconfig symbol, and drops earlier ARM64 constraints to enable
>> logging at architecture layer.
>>
>> Signed-off-by: Mario Smarduch <m.smarduch@samsung.com>
> 
> Just reminding you again of what I said in the previous thread (think
> that was before you sent this out), that you need to handle the pud_huge
> case in arch/arm/kvm/mmu.c for ARMv8 here.
> 
> -Christoffer
> 


  Yes, so like similar handling to what unmap_puds() does when
it encounters a PUD Block?

Should next revision be rebased to 'queued' 3.18.0-rc2?

Thanks.

[PATCH v2 3/3] KVM: arm/arm64: Enable Dirty Page logging for ARMv8

[Christoffer Dall]

On Mon, Nov 24, 2014 at 01:22:16PM -0800, Mario Smarduch wrote:
> On 11/22/2014 12:02 PM, Christoffer Dall wrote:
> > On Sat, Nov 15, 2014 at 12:19:10AM -0800, m.smarduch at samsung.com wrote:
> >> From: Mario Smarduch <m.smarduch@samsung.com>
> >>
> >> This patch enables ARMv8 ditry page logging support. Plugs ARMv8 into generic
> >> layer through Kconfig symbol, and drops earlier ARM64 constraints to enable
> >> logging at architecture layer.
> >>
> >> Signed-off-by: Mario Smarduch <m.smarduch@samsung.com>
> > 
> > Just reminding you again of what I said in the previous thread (think
> > that was before you sent this out), that you need to handle the pud_huge
> > case in arch/arm/kvm/mmu.c for ARMv8 here.
> > 
> > -Christoffer
> > 
> 
> 
>   Yes, so like similar handling to what unmap_puds() does when
> it encounters a PUD Block?

yes

> 
> Should next revision be rebased to 'queued' 3.18.0-rc2?
> 

rebase onto kvmarm/next whenever we move that to kvm/next after the
merge window (keep an eye out), typically it will be 3.19-rc1 + some
stuff from Paolo.

-Christoffer