Re: [PATCH] ARM: cache coherence problem in guestcopy.c

[Jaeyong Yoo <jaeyong.yoo@samsung.com>]

> 
> So I think we probably actually need the dcache flush in domain_map_page
> at the "/* Commandeer this 2MB slot */" point. In that context I don't
> think we can avoid flushing anything other than the complete 2MB
> mapping. Does this work for you too?

I am not sure that this would work. If we map_domain_page and unmap_domain_page
with the same mfn over and over again while the ref count is not zero (say 5), 
then flush is not called. And, I think we should call flush according to the reason below:

> 
> The laziness of the remappings makes me wonder though. Do you know if
> the slot is reused between step #2 and #3?  Otherwise I'd expect us to
> reuse the existing mapping with the cache intact. The caches are PIPT so
> I wouldn't expect the address aliasing to be an issue. Unless the
> mapping is reused for something else I'm not too sure where the cache
> pollution is coming from.

Let me try explain in more detail. 
We can consider the DomU as a producer (writing values to mfn) and
hypervisor as a consumer (reading values from mfn). While DomU is 
invoking multiple hypercalls, it reuses the same mfn and the same 
mapping at xen page table.

       (consumer)             (producer)
           xen                   DomU
             \                    /   (writing path)
            (cache)              /
                 \              /
(reading path)    \           /
_______________________________________
                    |   mfn   |        (physical memory)
---------------------------------------

If we see the above figure, xen "may" keep reading the cached value while 
DomU is writing different values to mfn. Here goes my observation where
cache pollution happen:

The pollution actually happens in second line of cache.
DomU side hypercall param local address is 0xc785fe38 (cache line size = 0x40B),
and the size of hypercall param is 24B. So the hypercall param lays out in two
cache lines. When hypervisor is reading the hypercall param, it reads the first 
8 bytes correctly (means the first cache line is flushed) and the other 16 bytes
are polluted (means the second cache line is not flushed).
Honestly, I'm not sure why the first cache line is flushed and the second is not.
I think we can also cache_line_align the hypercall param struct, but that is only
when  the sizes of all hypercall params are smaller than cache line size.

I hope the alignment of the figure is not broken :)

Best,
Jaeyong