[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[Stefano Stabellini]

Hi all,
Xen support in Linux for ARM and ARM64 suffers from lack of support for
multiple mfn to pfn mappings: whenever a frontend grants the same page
multiple times to the backend, the mfn to pfn accounting in
arch/arm/xen/p2m.c fails. The issue has become critical since v3.15,
when xen-netfront/xen-netback switched from grant copies to grant
mappings, therefore causing the issue to happen much more often.

Fixing the mfn to pfn accounting in p2m.c is difficult and expensive,
therefore we are looking for alternative solutions. One idea is avoiding
mfn to pfn conversions altogether. The only code path that needs them is
swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).

To avoid mfn to pfn conversions we rely on a second p2m mapping done by
Xen (a separate patch series will be sent for Xen). In Linux we use it
to perform the cache maintenance operations without mfns conversions.


The Xen side patch series that introduces XENFEAT_grant_map_identity is:
http://marc.info/?l=xen-devel&m=140690433923855




Changes in v3:
- rename XENFEAT_grant_map_11 to XENFEAT_grant_map_identity.

Changes in v2:
- introduce XENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need
to calculate it again every time;
- do not acutally unmap the page on xen_mm32_unmap;
- properly account preempt_enable/disable;
- do not check for mfn in xen_add_phys_to_mach_entry.



Stefano Stabellini (3):
      xen/arm: introduce XENFEAT_grant_map_identity
      xen/arm: reimplement xen_dma_unmap_page & friends
      xen/arm: remove mach_to_phys rbtree

 arch/arm/include/asm/xen/page-coherent.h |   25 ++--
 arch/arm/include/asm/xen/page.h          |    9 --
 arch/arm/xen/Makefile                    |    2 +-
 arch/arm/xen/enlighten.c                 |    6 +
 arch/arm/xen/mm32.c                      |  202 ++++++++++++++++++++++++++++++
 arch/arm/xen/p2m.c                       |   66 +---------
 include/xen/interface/features.h         |    3 +
 7 files changed, 220 insertions(+), 93 deletions(-)
 create mode 100644 arch/arm/xen/mm32.c

[PATCH v3 1/3] xen/arm: introduce XENFEAT_grant_map_identity

[Stefano Stabellini]

The flag tells us that the hypervisor maps a grant page to guest
physical address == machine address of the page in addition to the
normal grant mapping address. It is needed to properly issue cache
maintenance operation at the completion of a DMA operation involving a
foreign grant.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: Denis Schneider <v1ne2go@gmail.com>

---
Changes in v3:
- rename XENFEAT_grant_map_11 to XENFEAT_grant_map_identity.
---
 arch/arm/xen/enlighten.c         |    6 ++++++
 include/xen/interface/features.h |    3 +++
 2 files changed, 9 insertions(+)

diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
index b96723e..eef324f 100644
--- a/arch/arm/xen/enlighten.c
+++ b/arch/arm/xen/enlighten.c
@@ -262,6 +262,12 @@ static int __init xen_guest_init(void)
 	xen_domain_type = XEN_HVM_DOMAIN;
 
 	xen_setup_features();
+
+	if (!xen_feature(XENFEAT_grant_map_identity)) {
+		pr_warn("Please upgrade your Xen.\n"
+				"If your platform has any non-coherent DMA devices, they won't work properly.\n");
+	}
+
 	if (xen_feature(XENFEAT_dom0))
 		xen_start_info->flags |= SIF_INITDOMAIN|SIF_PRIVILEGED;
 	else
diff --git a/include/xen/interface/features.h b/include/xen/interface/features.h
index 131a6cc..14334d0 100644
--- a/include/xen/interface/features.h
+++ b/include/xen/interface/features.h
@@ -53,6 +53,9 @@
 /* operation as Dom0 is supported */
 #define XENFEAT_dom0                      11
 
+/* Xen also maps grant references at pfn = mfn */
+#define XENFEAT_grant_map_identity        12
+
 #define XENFEAT_NR_SUBMAPS 1
 
 #endif /* __XEN_PUBLIC_FEATURES_H__ */
-- 
1.7.10.4

[PATCH v3 1/3] xen/arm: introduce XENFEAT_grant_map_identity

[David Vrabel]

On 01/08/14 15:56, Stefano Stabellini wrote:
> The flag tells us that the hypervisor maps a grant page to guest
> physical address == machine address of the page in addition to the
> normal grant mapping address. It is needed to properly issue cache
> maintenance operation at the completion of a DMA operation involving a
> foreign grant.
> 
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Tested-by: Denis Schneider <v1ne2go@gmail.com>
> 
> ---
> Changes in v3:
> - rename XENFEAT_grant_map_11 to XENFEAT_grant_map_identity.
> ---
>  arch/arm/xen/enlighten.c         |    6 ++++++
>  include/xen/interface/features.h |    3 +++
>  2 files changed, 9 insertions(+)
> 
> diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
> index b96723e..eef324f 100644
> --- a/arch/arm/xen/enlighten.c
> +++ b/arch/arm/xen/enlighten.c
> @@ -262,6 +262,12 @@ static int __init xen_guest_init(void)
>  	xen_domain_type = XEN_HVM_DOMAIN;
>  
>  	xen_setup_features();
> +
> +	if (!xen_feature(XENFEAT_grant_map_identity)) {
> +		pr_warn("Please upgrade your Xen.\n"
> +				"If your platform has any non-coherent DMA devices, they won't work properly.\n");

I think:

"Non-coherent DMA devices require a hypervisor with
 XENFEAT_grant_map_identity"

Is a more helpful message.

David

[Xen-devel] [PATCH v3 1/3] xen/arm: introduce XENFEAT_grant_map_identity

[Julien Grall]

Hi Stefano,

On 01/08/14 15:56, Stefano Stabellini wrote:
> The flag tells us that the hypervisor maps a grant page to guest
> physical address == machine address of the page in addition to the
> normal grant mapping address. It is needed to properly issue cache
> maintenance operation at the completion of a DMA operation involving a
> foreign grant.
>
> Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Tested-by: Denis Schneider <v1ne2go@gmail.com>
>
> ---
> Changes in v3:
> - rename XENFEAT_grant_map_11 to XENFEAT_grant_map_identity.
> ---
>   arch/arm/xen/enlighten.c         |    6 ++++++
>   include/xen/interface/features.h |    3 +++
>   2 files changed, 9 insertions(+)
>
> diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
> index b96723e..eef324f 100644
> --- a/arch/arm/xen/enlighten.c
> +++ b/arch/arm/xen/enlighten.c
> @@ -262,6 +262,12 @@ static int __init xen_guest_init(void)
>   	xen_domain_type = XEN_HVM_DOMAIN;
>
>   	xen_setup_features();
> +
> +	if (!xen_feature(XENFEAT_grant_map_identity)) {
> +		pr_warn("Please upgrade your Xen.\n"
> +				"If your platform has any non-coherent DMA devices, they won't work properly.\n");
> +	}

I know we talked about this on a previous thread but I can't find it in 
my mail :/

Does a Linux with this series will work on Xen 4.4? If not what's the plan?

I think we have to keep any OS working on top on Xen 4.4 as people may 
want to retrieve the latest kernel for drivers bug fixes...

For running a previous kernel on Xen 4.5, then it will still be buggy. 
It might be worse to backport thise series on LTS kernel (i.e at least 
3.14).

Regards,

-- 
Julien Grall

[Xen-devel] [PATCH v3 1/3] xen/arm: introduce XENFEAT_grant_map_identity

[Stefano Stabellini]

On Fri, 1 Aug 2014, Julien Grall wrote:
> Hi Stefano,
> 
> On 01/08/14 15:56, Stefano Stabellini wrote:
> > The flag tells us that the hypervisor maps a grant page to guest
> > physical address == machine address of the page in addition to the
> > normal grant mapping address. It is needed to properly issue cache
> > maintenance operation at the completion of a DMA operation involving a
> > foreign grant.
> > 
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> > Tested-by: Denis Schneider <v1ne2go@gmail.com>
> > 
> > ---
> > Changes in v3:
> > - rename XENFEAT_grant_map_11 to XENFEAT_grant_map_identity.
> > ---
> >   arch/arm/xen/enlighten.c         |    6 ++++++
> >   include/xen/interface/features.h |    3 +++
> >   2 files changed, 9 insertions(+)
> > 
> > diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
> > index b96723e..eef324f 100644
> > --- a/arch/arm/xen/enlighten.c
> > +++ b/arch/arm/xen/enlighten.c
> > @@ -262,6 +262,12 @@ static int __init xen_guest_init(void)
> >   	xen_domain_type = XEN_HVM_DOMAIN;
> > 
> >   	xen_setup_features();
> > +
> > +	if (!xen_feature(XENFEAT_grant_map_identity)) {
> > +		pr_warn("Please upgrade your Xen.\n"
> > +				"If your platform has any non-coherent DMA
> > devices, they won't work properly.\n");
> > +	}
> 
> I know we talked about this on a previous thread but I can't find it in my
> mail :/
> 
> Does a Linux with this series will work on Xen 4.4? If not what's the plan?
 
I suggest we backport the changes to Xen 4.4.

"Still working" doesn't apply to a system without networking :-(


> I think we have to keep any OS working on top on Xen 4.4 as people may want to
> retrieve the latest kernel for drivers bug fixes...

Right, that's why we need to backport the fixes. Without the backport of
the Xen patches, with or without this Linux series, networking won't
work once the kernel is upgraded.


> For running a previous kernel on Xen 4.5, then it will still be buggy. It
> might be worse to backport thise series on LTS kernel (i.e at least 3.14).

The old thread is: http://marc.info/?l=xen-devel&m=140483414203686&w=2

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Stefano Stabellini]

xen_dma_unmap_page, xen_dma_sync_single_for_cpu and
xen_dma_sync_single_for_device are currently implemented by calling into
the corresponding generic ARM implementation of these functions. In
order to do this, firstly the dma_addr_t handle, that on Xen is a
machine address, needs to be translated into a physical address.  The
operation is expensive and inaccurate, given that a single machine
address can correspond to multiple physical addresses in one domain,
because the same page can be granted multiple times by the frontend.

To avoid this problem, we introduce a Xen specific implementation of
xen_dma_unmap_page, xen_dma_sync_single_for_cpu and
xen_dma_sync_single_for_device, that can operate on machine addresses
directly.

The new implementation relies on the fact that the hypervisor creates a
second p2m mapping of any grant pages at physical address == machine
address of the page for dom0. Therefore we can access memory at physical
address == dma_addr_r handle and perform the cache flushing there. Some
cache maintenance operations require a virtual address. Instead of using
ioremap_cache, that is not safe in interrupt context, we allocate a
per-cpu PAGE_KERNEL scratch page and we manually update the pte for it.

arm64 doesn't need cache maintenance operations on unmap for now.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: Denis Schneider <v1ne2go@gmail.com>

---

Changes in v3:
- rename XENFEAT_grant_map_11 to XENFEAT_grant_map_identity.

Changes in v2:
- check for XENFEAT_grant_map_11;
- remeber the ptep corresponding to scratch pages so that we don't need
to calculate it again every time;
- do not acutally unmap the page on xen_mm32_unmap;
- properly account preempt_enable/disable.
---
 arch/arm/include/asm/xen/page-coherent.h |   25 ++--
 arch/arm/xen/Makefile                    |    2 +-
 arch/arm/xen/mm32.c                      |  202 ++++++++++++++++++++++++++++++
 3 files changed, 210 insertions(+), 19 deletions(-)
 create mode 100644 arch/arm/xen/mm32.c

diff --git a/arch/arm/include/asm/xen/page-coherent.h b/arch/arm/include/asm/xen/page-coherent.h
index 1109017..e8275ea 100644
--- a/arch/arm/include/asm/xen/page-coherent.h
+++ b/arch/arm/include/asm/xen/page-coherent.h
@@ -26,25 +26,14 @@ static inline void xen_dma_map_page(struct device *hwdev, struct page *page,
 	__generic_dma_ops(hwdev)->map_page(hwdev, page, offset, size, dir, attrs);
 }
 
-static inline void xen_dma_unmap_page(struct device *hwdev, dma_addr_t handle,
+void xen_dma_unmap_page(struct device *hwdev, dma_addr_t handle,
 		size_t size, enum dma_data_direction dir,
-		struct dma_attrs *attrs)
-{
-	if (__generic_dma_ops(hwdev)->unmap_page)
-		__generic_dma_ops(hwdev)->unmap_page(hwdev, handle, size, dir, attrs);
-}
+		struct dma_attrs *attrs);
 
-static inline void xen_dma_sync_single_for_cpu(struct device *hwdev,
-		dma_addr_t handle, size_t size, enum dma_data_direction dir)
-{
-	if (__generic_dma_ops(hwdev)->sync_single_for_cpu)
-		__generic_dma_ops(hwdev)->sync_single_for_cpu(hwdev, handle, size, dir);
-}
+void xen_dma_sync_single_for_cpu(struct device *hwdev,
+		dma_addr_t handle, size_t size, enum dma_data_direction dir);
+
+void xen_dma_sync_single_for_device(struct device *hwdev,
+		dma_addr_t handle, size_t size, enum dma_data_direction dir);
 
-static inline void xen_dma_sync_single_for_device(struct device *hwdev,
-		dma_addr_t handle, size_t size, enum dma_data_direction dir)
-{
-	if (__generic_dma_ops(hwdev)->sync_single_for_device)
-		__generic_dma_ops(hwdev)->sync_single_for_device(hwdev, handle, size, dir);
-}
 #endif /* _ASM_ARM_XEN_PAGE_COHERENT_H */
diff --git a/arch/arm/xen/Makefile b/arch/arm/xen/Makefile
index 1296952..1f85bfe 100644
--- a/arch/arm/xen/Makefile
+++ b/arch/arm/xen/Makefile
@@ -1 +1 @@
-obj-y		:= enlighten.o hypercall.o grant-table.o p2m.o mm.o
+obj-y		:= enlighten.o hypercall.o grant-table.o p2m.o mm.o mm32.o
diff --git a/arch/arm/xen/mm32.c b/arch/arm/xen/mm32.c
new file mode 100644
index 0000000..534f362
--- /dev/null
+++ b/arch/arm/xen/mm32.c
@@ -0,0 +1,202 @@
+#include <linux/cpu.h>
+#include <linux/dma-mapping.h>
+#include <linux/gfp.h>
+#include <linux/highmem.h>
+
+#include <xen/features.h>
+
+static DEFINE_PER_CPU(unsigned long, xen_mm32_scratch_virt);
+static DEFINE_PER_CPU(pte_t *, xen_mm32_scratch_ptep);
+
+static int alloc_xen_mm32_scratch_page(int cpu)
+{
+	struct page *page;
+	unsigned long virt;
+	pmd_t *pmdp;
+	pte_t *ptep;
+
+	if (per_cpu(xen_mm32_scratch_ptep, cpu) != NULL)
+		return 0;
+
+	page = alloc_page(GFP_KERNEL);
+	if (page == NULL) {
+		pr_warn("Failed to allocate xen_mm32_scratch_page for cpu %d\n", cpu);
+		return -ENOMEM;
+	}
+
+	virt = (unsigned long)__va(page_to_phys(page));
+	pmdp = pmd_offset(pud_offset(pgd_offset_k(virt), virt), virt);
+	ptep = pte_offset_kernel(pmdp, virt);
+
+	per_cpu(xen_mm32_scratch_virt, cpu) = virt;
+	per_cpu(xen_mm32_scratch_ptep, cpu) = ptep;
+
+	return 0;
+}
+
+static int xen_mm32_cpu_notify(struct notifier_block *self,
+				    unsigned long action, void *hcpu)
+{
+	int cpu = (long)hcpu;
+	switch (action) {
+	case CPU_UP_PREPARE:
+		if (alloc_xen_mm32_scratch_page(cpu))
+			return NOTIFY_BAD;
+		break;
+	default:
+		break;
+	}
+	return NOTIFY_OK;
+}
+
+static struct notifier_block xen_mm32_cpu_notifier = {
+	.notifier_call	= xen_mm32_cpu_notify,
+};
+
+static void* xen_mm32_remap_page(dma_addr_t handle)
+{
+	unsigned long virt = get_cpu_var(xen_mm32_scratch_virt);
+	pte_t *ptep = __get_cpu_var(xen_mm32_scratch_ptep);
+
+	*ptep = pfn_pte(handle >> PAGE_SHIFT, PAGE_KERNEL);
+	local_flush_tlb_kernel_page(virt);
+
+	return (void*)virt;
+}
+
+static void xen_mm32_unmap(void *vaddr)
+{
+	put_cpu_var(xen_mm32_scratch_virt);
+}
+
+
+/* functions called by SWIOTLB */
+
+static void dma_cache_maint(dma_addr_t handle, unsigned long offset,
+	size_t size, enum dma_data_direction dir,
+	void (*op)(const void *, size_t, int))
+{
+	unsigned long pfn;
+	size_t left = size;
+
+	pfn = (handle >> PAGE_SHIFT) + offset / PAGE_SIZE;
+	offset %= PAGE_SIZE;
+
+	do {
+		size_t len = left;
+		void *vaddr;
+	
+		if (!pfn_valid(pfn))
+		{
+			/* Cannot map the page, we don't know its physical address.
+			 * Return and hope for the best */
+			if (!xen_feature(XENFEAT_grant_map_identity))
+				return;
+			vaddr = xen_mm32_remap_page(handle) + offset;
+			op(vaddr, len, dir);
+			xen_mm32_unmap(vaddr - offset);
+		} else {
+			struct page *page = pfn_to_page(pfn);
+
+			if (PageHighMem(page)) {
+				if (len + offset > PAGE_SIZE)
+					len = PAGE_SIZE - offset;
+
+				if (cache_is_vipt_nonaliasing()) {
+					vaddr = kmap_atomic(page);
+					op(vaddr + offset, len, dir);
+					kunmap_atomic(vaddr);
+				} else {
+					vaddr = kmap_high_get(page);
+					if (vaddr) {
+						op(vaddr + offset, len, dir);
+						kunmap_high(page);
+					}
+				}
+			} else {
+				vaddr = page_address(page) + offset;
+				op(vaddr, len, dir);
+			}
+		}
+
+		offset = 0;
+		pfn++;
+		left -= len;
+	} while (left);
+}
+
+static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
+		size_t size, enum dma_data_direction dir)
+{
+	/* Cannot use __dma_page_dev_to_cpu because we don't have a
+	 * struct page for handle */
+
+	if (dir == DMA_TO_DEVICE)
+		outer_inv_range(handle, handle + size);
+
+	dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, dmac_unmap_area);
+}
+
+static void __xen_dma_page_cpu_to_dev(struct device *hwdev, dma_addr_t handle,
+		size_t size, enum dma_data_direction dir)
+{
+
+	dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, dmac_map_area);
+
+	if (dir == DMA_FROM_DEVICE) {
+		outer_inv_range(handle, handle + size);
+	} else {
+		outer_clean_range(handle, handle + size);
+	}
+}
+
+void xen_dma_unmap_page(struct device *hwdev, dma_addr_t handle,
+		size_t size, enum dma_data_direction dir,
+		struct dma_attrs *attrs)
+
+{
+	if (!__generic_dma_ops(hwdev)->unmap_page)
+		return;
+	if (dma_get_attr(DMA_ATTR_SKIP_CPU_SYNC, attrs))
+		return;
+
+	__xen_dma_page_dev_to_cpu(hwdev, handle, size, dir);
+}
+
+void xen_dma_sync_single_for_cpu(struct device *hwdev,
+		dma_addr_t handle, size_t size, enum dma_data_direction dir)
+{
+	if (!__generic_dma_ops(hwdev)->sync_single_for_cpu)
+		return;
+	__xen_dma_page_dev_to_cpu(hwdev, handle, size, dir);
+}
+
+void xen_dma_sync_single_for_device(struct device *hwdev,
+		dma_addr_t handle, size_t size, enum dma_data_direction dir)
+{
+	if (!__generic_dma_ops(hwdev)->sync_single_for_device)
+		return;
+	__xen_dma_page_cpu_to_dev(hwdev, handle, size, dir);
+}
+
+int __init xen_mm32_init(void)
+{
+	int cpu;
+
+	if (!xen_initial_domain())
+		return 0;
+
+	register_cpu_notifier(&xen_mm32_cpu_notifier);
+	get_online_cpus();
+	for_each_online_cpu(cpu) {
+		if (alloc_xen_mm32_scratch_page(cpu)) {
+			put_online_cpus();
+			unregister_cpu_notifier(&xen_mm32_cpu_notifier);
+			return -ENOMEM;
+		}
+	}
+	put_online_cpus();
+
+	return 0;
+}
+arch_initcall(xen_mm32_init);
-- 
1.7.10.4

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Stefano Stabellini]

On Fri, 1 Aug 2014, Stefano Stabellini wrote:
> +static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
> +		size_t size, enum dma_data_direction dir)
> +{
> +	/* Cannot use __dma_page_dev_to_cpu because we don't have a
> +	 * struct page for handle */
> +
> +	if (dir == DMA_TO_DEVICE)

This should be:
    if (dir != DMA_TO_DEVICE)

Thomas, could you please confirm that with this small fix
http://pastebin.com/FPRf7pgL goes away?


> +		outer_inv_range(handle, handle + size);
> +
> +	dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, dmac_unmap_area);
> +}

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Wei Liu]

On Fri, Aug 08, 2014 at 03:32:41PM +0100, Stefano Stabellini wrote:
> On Fri, 1 Aug 2014, Stefano Stabellini wrote:
> > +static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
> > +		size_t size, enum dma_data_direction dir)
> > +{
> > +	/* Cannot use __dma_page_dev_to_cpu because we don't have a
> > +	 * struct page for handle */
> > +
> > +	if (dir == DMA_TO_DEVICE)
> 
> This should be:
>     if (dir != DMA_TO_DEVICE)
> 
> Thomas, could you please confirm that with this small fix
> http://pastebin.com/FPRf7pgL goes away?
> 

Thomas, please try this fix with my ref-counting patch.

The old "working" version might actually cover this latent bug due to
it's long delay.

Wei.

> 
> > +		outer_inv_range(handle, handle + size);
> > +
> > +	dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, dmac_unmap_area);
> > +}

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Thomas Leonard]

On 8 August 2014 15:38, Wei Liu <wei.liu2@citrix.com> wrote:
> On Fri, Aug 08, 2014 at 03:32:41PM +0100, Stefano Stabellini wrote:
>> On Fri, 1 Aug 2014, Stefano Stabellini wrote:
>> > +static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
>> > +           size_t size, enum dma_data_direction dir)
>> > +{
>> > +   /* Cannot use __dma_page_dev_to_cpu because we don't have a
>> > +    * struct page for handle */
>> > +
>> > +   if (dir == DMA_TO_DEVICE)
>>
>> This should be:
>>     if (dir != DMA_TO_DEVICE)
>>
>> Thomas, could you please confirm that with this small fix
>> http://pastebin.com/FPRf7pgL goes away?
>>
>
> Thomas, please try this fix with my ref-counting patch.
>
> The old "working" version might actually cover this latent bug due to
> it's long delay.

I'm not sure how to apply this. The function
"__xen_dma_page_dev_to_cpu" doesn't appear in your "for-thomas"
branch. If you push the change to that branch I can test it.

> Wei.
>
>>
>> > +           outer_inv_range(handle, handle + size);
>> > +
>> > +   dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, dmac_unmap_area);
>> > +}



-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Stefano Stabellini]

On Fri, 8 Aug 2014, Thomas Leonard wrote:
> On 8 August 2014 15:38, Wei Liu <wei.liu2@citrix.com> wrote:
> > On Fri, Aug 08, 2014 at 03:32:41PM +0100, Stefano Stabellini wrote:
> >> On Fri, 1 Aug 2014, Stefano Stabellini wrote:
> >> > +static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
> >> > +           size_t size, enum dma_data_direction dir)
> >> > +{
> >> > +   /* Cannot use __dma_page_dev_to_cpu because we don't have a
> >> > +    * struct page for handle */
> >> > +
> >> > +   if (dir == DMA_TO_DEVICE)
> >>
> >> This should be:
> >>     if (dir != DMA_TO_DEVICE)
> >>
> >> Thomas, could you please confirm that with this small fix
> >> http://pastebin.com/FPRf7pgL goes away?
> >>
> >
> > Thomas, please try this fix with my ref-counting patch.
> >
> > The old "working" version might actually cover this latent bug due to
> > it's long delay.
> 
> I'm not sure how to apply this. The function
> "__xen_dma_page_dev_to_cpu" doesn't appear in your "for-thomas"
> branch. If you push the change to that branch I can test it.

That's the patch for you:

diff --git a/arch/arm/xen/mm32.c b/arch/arm/xen/mm32.c
index 534f362..3b99860 100644
--- a/arch/arm/xen/mm32.c
+++ b/arch/arm/xen/mm32.c
@@ -131,7 +131,7 @@ static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
 	/* Cannot use __dma_page_dev_to_cpu because we don't have a
 	 * struct page for handle */
 
-	if (dir == DMA_TO_DEVICE)
+	if (dir != DMA_TO_DEVICE)
 		outer_inv_range(handle, handle + size);
 
 	dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, dmac_unmap_area);

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Wei Liu]

On Fri, Aug 08, 2014 at 03:49:26PM +0100, Thomas Leonard wrote:
> On 8 August 2014 15:38, Wei Liu <wei.liu2@citrix.com> wrote:
> > On Fri, Aug 08, 2014 at 03:32:41PM +0100, Stefano Stabellini wrote:
> >> On Fri, 1 Aug 2014, Stefano Stabellini wrote:
> >> > +static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
> >> > +           size_t size, enum dma_data_direction dir)
> >> > +{
> >> > +   /* Cannot use __dma_page_dev_to_cpu because we don't have a
> >> > +    * struct page for handle */
> >> > +
> >> > +   if (dir == DMA_TO_DEVICE)
> >>
> >> This should be:
> >>     if (dir != DMA_TO_DEVICE)
> >>
> >> Thomas, could you please confirm that with this small fix
> >> http://pastebin.com/FPRf7pgL goes away?
> >>
> >
> > Thomas, please try this fix with my ref-counting patch.
> >
> > The old "working" version might actually cover this latent bug due to
> > it's long delay.
> 
> I'm not sure how to apply this. The function
> "__xen_dma_page_dev_to_cpu" doesn't appear in your "for-thomas"
> branch. If you push the change to that branch I can test it.
> 

I think you can cherry-pick my three patches to your tree which contains
Stefano's patches. It's probably easier because Stefano's patches are
not yet in mainline while my patches should be able to apply to 3.16
mainline kernel without much effort.

I've rebased my patches on top of 3.16, in for-thomas2 branch.

Wei.

> > Wei.
> >
> >>
> >> > +           outer_inv_range(handle, handle + size);
> >> > +
> >> > +   dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, dmac_unmap_area);
> >> > +}
> 
> 
> 
> -- 
> Dr Thomas Leonard        http://0install.net/
> GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
> GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Thomas Leonard]

On 8 August 2014 16:00, Wei Liu <wei.liu2@citrix.com> wrote:
> On Fri, Aug 08, 2014 at 03:49:26PM +0100, Thomas Leonard wrote:
>> On 8 August 2014 15:38, Wei Liu <wei.liu2@citrix.com> wrote:
>> > On Fri, Aug 08, 2014 at 03:32:41PM +0100, Stefano Stabellini wrote:
>> >> On Fri, 1 Aug 2014, Stefano Stabellini wrote:
>> >> > +static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
>> >> > +           size_t size, enum dma_data_direction dir)
>> >> > +{
>> >> > +   /* Cannot use __dma_page_dev_to_cpu because we don't have a
>> >> > +    * struct page for handle */
>> >> > +
>> >> > +   if (dir == DMA_TO_DEVICE)
>> >>
>> >> This should be:
>> >>     if (dir != DMA_TO_DEVICE)
>> >>
>> >> Thomas, could you please confirm that with this small fix
>> >> http://pastebin.com/FPRf7pgL goes away?
>> >>
>> >
>> > Thomas, please try this fix with my ref-counting patch.
>> >
>> > The old "working" version might actually cover this latent bug due to
>> > it's long delay.
>>
>> I'm not sure how to apply this. The function
>> "__xen_dma_page_dev_to_cpu" doesn't appear in your "for-thomas"
>> branch. If you push the change to that branch I can test it.
>>
>
> I think you can cherry-pick my three patches to your tree which contains
> Stefano's patches. It's probably easier because Stefano's patches are
> not yet in mainline while my patches should be able to apply to 3.16
> mainline kernel without much effort.
>
> I've rebased my patches on top of 3.16, in for-thomas2 branch.

OK, that works for me! I tried starting the quickly-exiting VM a few
times and it didn't break anything, and my queue service served up the
file too. Thanks!

(for reference, this is the exact commit I tested:
https://github.com/talex5/linux/commit/026c61c9649e0ef30546222a429a1b11ae7a7555)


-- 
Dr Thomas Leonard        http://0install.net/
GPG: 9242 9807 C985 3C07 44A6  8B9A AE07 8280 59A5 3CC1
GPG: DA98 25AE CAD0 8975 7CDA  BD8E 0713 3F96 CA74 D8BA

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Wei Liu]

On Fri, Aug 08, 2014 at 05:01:42PM +0100, Thomas Leonard wrote:
[...]
> OK, that works for me! I tried starting the quickly-exiting VM a few
> times and it didn't break anything, and my queue service served up the
> file too. Thanks!
> 
> (for reference, this is the exact commit I tested:
> https://github.com/talex5/linux/commit/026c61c9649e0ef30546222a429a1b11ae7a7555)
> 

Thanks for testing.

I will submit my network driver patches shortly.

Wei.

[PATCH v3 2/3] xen/arm: reimplement xen_dma_unmap_page & friends

[Stefano Stabellini]

On Fri, 8 Aug 2014, Thomas Leonard wrote:
> On 8 August 2014 16:00, Wei Liu <wei.liu2@citrix.com> wrote:
> > On Fri, Aug 08, 2014 at 03:49:26PM +0100, Thomas Leonard wrote:
> >> On 8 August 2014 15:38, Wei Liu <wei.liu2@citrix.com> wrote:
> >> > On Fri, Aug 08, 2014 at 03:32:41PM +0100, Stefano Stabellini wrote:
> >> >> On Fri, 1 Aug 2014, Stefano Stabellini wrote:
> >> >> > +static void __xen_dma_page_dev_to_cpu(struct device *hwdev, dma_addr_t handle,
> >> >> > +           size_t size, enum dma_data_direction dir)
> >> >> > +{
> >> >> > +   /* Cannot use __dma_page_dev_to_cpu because we don't have a
> >> >> > +    * struct page for handle */
> >> >> > +
> >> >> > +   if (dir == DMA_TO_DEVICE)
> >> >>
> >> >> This should be:
> >> >>     if (dir != DMA_TO_DEVICE)
> >> >>
> >> >> Thomas, could you please confirm that with this small fix
> >> >> http://pastebin.com/FPRf7pgL goes away?
> >> >>
> >> >
> >> > Thomas, please try this fix with my ref-counting patch.
> >> >
> >> > The old "working" version might actually cover this latent bug due to
> >> > it's long delay.
> >>
> >> I'm not sure how to apply this. The function
> >> "__xen_dma_page_dev_to_cpu" doesn't appear in your "for-thomas"
> >> branch. If you push the change to that branch I can test it.
> >>
> >
> > I think you can cherry-pick my three patches to your tree which contains
> > Stefano's patches. It's probably easier because Stefano's patches are
> > not yet in mainline while my patches should be able to apply to 3.16
> > mainline kernel without much effort.
> >
> > I've rebased my patches on top of 3.16, in for-thomas2 branch.
> 
> OK, that works for me! I tried starting the quickly-exiting VM a few
> times and it didn't break anything, and my queue service served up the
> file too. Thanks!
> 
> (for reference, this is the exact commit I tested:
> https://github.com/talex5/linux/commit/026c61c9649e0ef30546222a429a1b11ae7a7555)
> 

Thanks for testing! I'll update my patch series with the small fix.

[PATCH v3 3/3] xen/arm: remove mach_to_phys rbtree

[Stefano Stabellini]

Remove the rbtree used to keep track of machine to physical mappings:
the frontend can grant the same page multiple times, leading to errors
inserting or removing entries from the mach_to_phys tree.

Linux only needed to know the physical address corresponding to a given
machine address in swiotlb-xen. Now that swiotlb-xen can call the
xen_dma_* functions passing the machine address directly, we can remove
it.

Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tested-by: Denis Schneider <v1ne2go@gmail.com>

---

Changes in v2:
- do not check for mfn in xen_add_phys_to_mach_entry.
---
 arch/arm/include/asm/xen/page.h |    9 ------
 arch/arm/xen/p2m.c              |   66 +--------------------------------------
 2 files changed, 1 insertion(+), 74 deletions(-)

diff --git a/arch/arm/include/asm/xen/page.h b/arch/arm/include/asm/xen/page.h
index ded062f..135c24a 100644
--- a/arch/arm/include/asm/xen/page.h
+++ b/arch/arm/include/asm/xen/page.h
@@ -33,7 +33,6 @@ typedef struct xpaddr {
 #define INVALID_P2M_ENTRY      (~0UL)
 
 unsigned long __pfn_to_mfn(unsigned long pfn);
-unsigned long __mfn_to_pfn(unsigned long mfn);
 extern struct rb_root phys_to_mach;
 
 static inline unsigned long pfn_to_mfn(unsigned long pfn)
@@ -51,14 +50,6 @@ static inline unsigned long pfn_to_mfn(unsigned long pfn)
 
 static inline unsigned long mfn_to_pfn(unsigned long mfn)
 {
-	unsigned long pfn;
-
-	if (phys_to_mach.rb_node != NULL) {
-		pfn = __mfn_to_pfn(mfn);
-		if (pfn != INVALID_P2M_ENTRY)
-			return pfn;
-	}
-
 	return mfn;
 }
 
diff --git a/arch/arm/xen/p2m.c b/arch/arm/xen/p2m.c
index 97baf44..0548577 100644
--- a/arch/arm/xen/p2m.c
+++ b/arch/arm/xen/p2m.c
@@ -21,14 +21,12 @@ struct xen_p2m_entry {
 	unsigned long pfn;
 	unsigned long mfn;
 	unsigned long nr_pages;
-	struct rb_node rbnode_mach;
 	struct rb_node rbnode_phys;
 };
 
 static rwlock_t p2m_lock;
 struct rb_root phys_to_mach = RB_ROOT;
 EXPORT_SYMBOL_GPL(phys_to_mach);
-static struct rb_root mach_to_phys = RB_ROOT;
 
 static int xen_add_phys_to_mach_entry(struct xen_p2m_entry *new)
 {
@@ -41,8 +39,6 @@ static int xen_add_phys_to_mach_entry(struct xen_p2m_entry *new)
 		parent = *link;
 		entry = rb_entry(parent, struct xen_p2m_entry, rbnode_phys);
 
-		if (new->mfn == entry->mfn)
-			goto err_out;
 		if (new->pfn == entry->pfn)
 			goto err_out;
 
@@ -88,64 +84,6 @@ unsigned long __pfn_to_mfn(unsigned long pfn)
 }
 EXPORT_SYMBOL_GPL(__pfn_to_mfn);
 
-static int xen_add_mach_to_phys_entry(struct xen_p2m_entry *new)
-{
-	struct rb_node **link = &mach_to_phys.rb_node;
-	struct rb_node *parent = NULL;
-	struct xen_p2m_entry *entry;
-	int rc = 0;
-
-	while (*link) {
-		parent = *link;
-		entry = rb_entry(parent, struct xen_p2m_entry, rbnode_mach);
-
-		if (new->mfn == entry->mfn)
-			goto err_out;
-		if (new->pfn == entry->pfn)
-			goto err_out;
-
-		if (new->mfn < entry->mfn)
-			link = &(*link)->rb_left;
-		else
-			link = &(*link)->rb_right;
-	}
-	rb_link_node(&new->rbnode_mach, parent, link);
-	rb_insert_color(&new->rbnode_mach, &mach_to_phys);
-	goto out;
-
-err_out:
-	rc = -EINVAL;
-	pr_warn("%s: cannot add pfn=%pa -> mfn=%pa: pfn=%pa -> mfn=%pa already exists\n",
-			__func__, &new->pfn, &new->mfn, &entry->pfn, &entry->mfn);
-out:
-	return rc;
-}
-
-unsigned long __mfn_to_pfn(unsigned long mfn)
-{
-	struct rb_node *n = mach_to_phys.rb_node;
-	struct xen_p2m_entry *entry;
-	unsigned long irqflags;
-
-	read_lock_irqsave(&p2m_lock, irqflags);
-	while (n) {
-		entry = rb_entry(n, struct xen_p2m_entry, rbnode_mach);
-		if (entry->mfn <= mfn &&
-				entry->mfn + entry->nr_pages > mfn) {
-			read_unlock_irqrestore(&p2m_lock, irqflags);
-			return entry->pfn + (mfn - entry->mfn);
-		}
-		if (mfn < entry->mfn)
-			n = n->rb_left;
-		else
-			n = n->rb_right;
-	}
-	read_unlock_irqrestore(&p2m_lock, irqflags);
-
-	return INVALID_P2M_ENTRY;
-}
-EXPORT_SYMBOL_GPL(__mfn_to_pfn);
-
 int set_foreign_p2m_mapping(struct gnttab_map_grant_ref *map_ops,
 			    struct gnttab_map_grant_ref *kmap_ops,
 			    struct page **pages, unsigned int count)
@@ -192,7 +130,6 @@ bool __set_phys_to_machine_multi(unsigned long pfn,
 			p2m_entry = rb_entry(n, struct xen_p2m_entry, rbnode_phys);
 			if (p2m_entry->pfn <= pfn &&
 					p2m_entry->pfn + p2m_entry->nr_pages > pfn) {
-				rb_erase(&p2m_entry->rbnode_mach, &mach_to_phys);
 				rb_erase(&p2m_entry->rbnode_phys, &phys_to_mach);
 				write_unlock_irqrestore(&p2m_lock, irqflags);
 				kfree(p2m_entry);
@@ -217,8 +154,7 @@ bool __set_phys_to_machine_multi(unsigned long pfn,
 	p2m_entry->mfn = mfn;
 
 	write_lock_irqsave(&p2m_lock, irqflags);
-	if ((rc = xen_add_phys_to_mach_entry(p2m_entry) < 0) ||
-		(rc = xen_add_mach_to_phys_entry(p2m_entry) < 0)) {
+	if ((rc = xen_add_phys_to_mach_entry(p2m_entry)) < 0) {
 		write_unlock_irqrestore(&p2m_lock, irqflags);
 		return false;
 	}
-- 
1.7.10.4

[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[David Vrabel]

On 01/08/14 15:55, Stefano Stabellini wrote:
> Hi all,
> Xen support in Linux for ARM and ARM64 suffers from lack of support for
> multiple mfn to pfn mappings: whenever a frontend grants the same page
> multiple times to the backend, the mfn to pfn accounting in
> arch/arm/xen/p2m.c fails. The issue has become critical since v3.15,
> when xen-netfront/xen-netback switched from grant copies to grant
> mappings, therefore causing the issue to happen much more often.
> 
> Fixing the mfn to pfn accounting in p2m.c is difficult and expensive,
> therefore we are looking for alternative solutions. One idea is avoiding
> mfn to pfn conversions altogether. The only code path that needs them is
> swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).
> 
> To avoid mfn to pfn conversions we rely on a second p2m mapping done by
> Xen (a separate patch series will be sent for Xen). In Linux we use it
> to perform the cache maintenance operations without mfns conversions.
> 
> 
> The Xen side patch series that introduces XENFEAT_grant_map_identity is:
> http://marc.info/?l=xen-devel&m=140690433923855

If the Xen side is in master, you can apply this series to
devel/for-linus-3.17

David

[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[Stefano Stabellini]

On Fri, 1 Aug 2014, David Vrabel wrote:
> On 01/08/14 15:55, Stefano Stabellini wrote:
> > Hi all,
> > Xen support in Linux for ARM and ARM64 suffers from lack of support for
> > multiple mfn to pfn mappings: whenever a frontend grants the same page
> > multiple times to the backend, the mfn to pfn accounting in
> > arch/arm/xen/p2m.c fails. The issue has become critical since v3.15,
> > when xen-netfront/xen-netback switched from grant copies to grant
> > mappings, therefore causing the issue to happen much more often.
> > 
> > Fixing the mfn to pfn accounting in p2m.c is difficult and expensive,
> > therefore we are looking for alternative solutions. One idea is avoiding
> > mfn to pfn conversions altogether. The only code path that needs them is
> > swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).
> > 
> > To avoid mfn to pfn conversions we rely on a second p2m mapping done by
> > Xen (a separate patch series will be sent for Xen). In Linux we use it
> > to perform the cache maintenance operations without mfns conversions.
> > 
> > 
> > The Xen side patch series that introduces XENFEAT_grant_map_identity is:
> > http://marc.info/?l=xen-devel&m=140690433923855
> 
> If the Xen side is in master, you can apply this series to
> devel/for-linus-3.17

Thanks. The Xen series is not in yet, I'll do as soon as it is applied.

[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[Stefano Stabellini]

On Fri, 1 Aug 2014, David Vrabel wrote:
> On 01/08/14 15:55, Stefano Stabellini wrote:
> > Hi all,
> > Xen support in Linux for ARM and ARM64 suffers from lack of support for
> > multiple mfn to pfn mappings: whenever a frontend grants the same page
> > multiple times to the backend, the mfn to pfn accounting in
> > arch/arm/xen/p2m.c fails. The issue has become critical since v3.15,
> > when xen-netfront/xen-netback switched from grant copies to grant
> > mappings, therefore causing the issue to happen much more often.
> > 
> > Fixing the mfn to pfn accounting in p2m.c is difficult and expensive,
> > therefore we are looking for alternative solutions. One idea is avoiding
> > mfn to pfn conversions altogether. The only code path that needs them is
> > swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).
> > 
> > To avoid mfn to pfn conversions we rely on a second p2m mapping done by
> > Xen (a separate patch series will be sent for Xen). In Linux we use it
> > to perform the cache maintenance operations without mfns conversions.
> > 
> > 
> > The Xen side patch series that introduces XENFEAT_grant_map_identity is:
> > http://marc.info/?l=xen-devel&m=140690433923855
> 
> If the Xen side is in master, you can apply this series to
> devel/for-linus-3.17

The Xen side patch finally went in. Unfortunately it's probably too late
for 3.17.

I have applied the latest version
(http://marc.info/?l=linux-arm-kernel&m=140751761520724&w=2) to
devel/for-linus-3.18.

[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[David Vrabel]

On 11/09/14 00:28, Stefano Stabellini wrote:
> On Fri, 1 Aug 2014, David Vrabel wrote:
>> On 01/08/14 15:55, Stefano Stabellini wrote:
>>> Hi all,
>>> Xen support in Linux for ARM and ARM64 suffers from lack of support for
>>> multiple mfn to pfn mappings: whenever a frontend grants the same page
>>> multiple times to the backend, the mfn to pfn accounting in
>>> arch/arm/xen/p2m.c fails. The issue has become critical since v3.15,
>>> when xen-netfront/xen-netback switched from grant copies to grant
>>> mappings, therefore causing the issue to happen much more often.
>>>
>>> Fixing the mfn to pfn accounting in p2m.c is difficult and expensive,
>>> therefore we are looking for alternative solutions. One idea is avoiding
>>> mfn to pfn conversions altogether. The only code path that needs them is
>>> swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).
>>>
>>> To avoid mfn to pfn conversions we rely on a second p2m mapping done by
>>> Xen (a separate patch series will be sent for Xen). In Linux we use it
>>> to perform the cache maintenance operations without mfns conversions.
>>>
>>>
>>> The Xen side patch series that introduces XENFEAT_grant_map_identity is:
>>> http://marc.info/?l=xen-devel&m=140690433923855
>>
>> If the Xen side is in master, you can apply this series to
>> devel/for-linus-3.17
> 
> The Xen side patch finally went in. Unfortunately it's probably too late
> for 3.17.

Isn't this a bug fix?  I think would be fine to go into 3.17.

David

[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[Stefano Stabellini]

On Thu, 11 Sep 2014, David Vrabel wrote:
> On 11/09/14 00:28, Stefano Stabellini wrote:
> > On Fri, 1 Aug 2014, David Vrabel wrote:
> >> On 01/08/14 15:55, Stefano Stabellini wrote:
> >>> Hi all,
> >>> Xen support in Linux for ARM and ARM64 suffers from lack of support for
> >>> multiple mfn to pfn mappings: whenever a frontend grants the same page
> >>> multiple times to the backend, the mfn to pfn accounting in
> >>> arch/arm/xen/p2m.c fails. The issue has become critical since v3.15,
> >>> when xen-netfront/xen-netback switched from grant copies to grant
> >>> mappings, therefore causing the issue to happen much more often.
> >>>
> >>> Fixing the mfn to pfn accounting in p2m.c is difficult and expensive,
> >>> therefore we are looking for alternative solutions. One idea is avoiding
> >>> mfn to pfn conversions altogether. The only code path that needs them is
> >>> swiotlb-xen:unmap_page (and single_for_cpu and single_for_device).
> >>>
> >>> To avoid mfn to pfn conversions we rely on a second p2m mapping done by
> >>> Xen (a separate patch series will be sent for Xen). In Linux we use it
> >>> to perform the cache maintenance operations without mfns conversions.
> >>>
> >>>
> >>> The Xen side patch series that introduces XENFEAT_grant_map_identity is:
> >>> http://marc.info/?l=xen-devel&m=140690433923855
> >>
> >> If the Xen side is in master, you can apply this series to
> >> devel/for-linus-3.17
> > 
> > The Xen side patch finally went in. Unfortunately it's probably too late
> > for 3.17.
> 
> Isn't this a bug fix?  I think would be fine to go into 3.17.
 
Yes, you are right, it is a bug fix. I was being too conservative.

I applied it to stable/for-linus-3.17-b

Do you have any more fixes queued up to send to Linus for 3.17?

[PATCH v3 0/3] xen/arm: fix "xen_add_mach_to_phys_entry: cannot add" problem

[David Vrabel]

On 11/09/14 19:31, Stefano Stabellini wrote:
> 
> Do you have any more fixes queued up to send to Linus for 3.17?

I sent a batch yesterday.  Sorry for not waiting for these -- I forgot
about them.

David