[ARM] Bash often segfaults in Dom0 with the latest Xen

[Julien Grall <julien.grall@linaro.org>]

Hi all,

Since a couple of week,  I'm tracking an issue with Xen on ARM with no luck.

I'm run out of idea, so I send this email to have advice from the community.

Most of the time bash will abort with random error in dom0:
  - page fault (data and prefetch abort)
  - memory corruption (malloc corruption and invalid pointer)

It's easily to reproduce by doing ./configure on the xen tree.

My environment is an arndale board:
  - linux linaro 13.05 (using arndale_xen_dom0_defconfig and exynos5250_arndale.dts)
  - opensuse 12.03 (http://en.opensuse.org/HCL:Arndale)
  - xen upstream

The linux tree can be retrieved from git://xenbits.xen.org/people/julieng/linux-arm.git
using the branch linaro-3.10.
The previous branch is based on the linaro tree with some patches for the dts and xen.

The issue also occurs on the versatile express. But it's harder to reproduce.
Here the environment is:
  - linux linaro 13.05 (using vexpress_xen_dom0_defconfig and vexpress_v2p_ca15_a7.dtb)
  - ubuntu linaro 13.05
  - xen upstream

I have tried different distributions and linux version, the issue was the same.
I made some testing to narrow down the bug and I came to the following test case:

Only dom0 is running and each VCPUs are pinned to a specific cpu
(vcpu0 -> cpu0 and vcpu1 -> cpu1).

The patch below removes WFI trap and by consequence avoid a VCPU to move to
another physical CPU.
=========================================
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 6cfba1a..e89ca15 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -62,7 +62,7 @@ void __cpuinit init_traps(void)
     WRITE_SYSREG((vaddr_t)hyp_traps_vector, VBAR_EL2);
 
     /* Setup hypervisor traps */
-    WRITE_SYSREG(HCR_PTW|HCR_BSU_OUTER|HCR_AMO|HCR_IMO|HCR_VM|HCR_TWI|HCR_TSC, HCR_EL2);
+    WRITE_SYSREG(HCR_PTW|HCR_BSU_OUTER|HCR_AMO|HCR_IMO|HCR_VM|HCR_TSC, HCR_EL2);
     isb();
 }
 
=========================================

If a bash process is assigned to a specific cpu with taskset, the process seems
to always run without any issue.

taskset -c 0 ./configure

I guess it's a caching issue, but each time I've tried to play with the caching
policy Linux was not booting.

Thanks in advance for any advice.

Cheers,

--
Julien Grall