Re: [PATCH V2 1/3] MIPS: Fix cache flushing for swap pages with non-DMA I/O.

[David Daney <ddaney.cavm@gmail.com>]

On 02/24/2015 04:07 PM, Maciej W. Rozycki wrote:
> On Tue, 24 Feb 2015, Leonid Yegoshin wrote:
>
>>> SYNCI is only useful in non-SMP kernels.
>> Yes, until MIPS R6. I pressed hard on Arch team to change vague words in SYNCI
>> description and now (MIPS R6) it has words requiring execution on all cores:
>>
>>> "SYNCI globalization:
>>> Release 6: SYNCI globalization (as described below) is required: compliant
>>> implementations must globalize SYNCI.
>>> Portable software can rely on this behavior, and use SYNCI rather than
>>> expensive “instruction cache shootdown”
>>> using inter-processor interrupts."
>
>   Good, thanks for enforcing sanity!
>
>>> If a thread is migrated to a different CPU between the SYNCI, and the
>>> attempt to execute the freshly generated code, the new CPU can still have a
>>> dirty ICACHE.  So for Linux userspace, cacheflush(2) is your only option.
>
>   Is it not a kernel bug then?

I don't think so. cacheflush(2) is the only way for the kernel to know 
that it needs to do extra work.  The alternative is to not use ASIDs in 
the TLB and caches, and just invalidate all caches whenever we change 
contexts.

We carry a lot complexity in the kernel to avoid unnecessary TLB and 
cache invalidations.  All that goes out the window (with the performance 
improvements that it brings) if you want to magically make cacheflush(2) 
unnecessary.

> Shouldn't migration code enforce cache
> coherency manually if hardware does not?  User software is supposed to
> have a consistent view of the system and such details as being run on a
> multiprocessor should be completely hidden.

Traditionally on MIPS, the ICache is not coherent, going from non-SMP to 
SMP doesn't magically make it coherent.  Part of the ABI is that if you 
have self modifying code, you must call cacheflush(2)  If you violate 
the ABI, it won't work.

David Daney