Re: Accessing emulated CXL memory is unstable

[Gregory Price <gregory.price@memverge.com>]

On Tue, Oct 10, 2023 at 10:35:03AM +0900, Hyeonggon Yoo wrote:
> Hello folks,
> 
> I experienced strange application crashes/internal KVM errors
> while playing with emulated type 3 CXL memory. I would like to know
> if this is a real issue or I missed something during setup.
> 
> TL;DR: applications crash when accessing emulated CXL memory,
> and stressing VM subsystem causes KVM internal error
> (stressing via stress-ng --bigheap)
>
...
> 
> Hmm... it crashed, and it's 'invalid opcode'.
> Is this because the fetched instruction is different from what's
> written to memory during exec()?
> 

This is a known issue, and the working theory is 2 issues:

1) CXL devices are implemented on top of an MMIO-style dispatch system
   and as a result memory from CXL is non-cacheable.  We think there
   may be an issue with this in KVM but it hasn't been investigated
   fully.

2) When we originally got CXL memory support, we discovered an edge case
   where code pages hosted on CXL memory would cause a crash whenever an
   instruction spanned across a page barrier.  A similar issue could
   affect KVM.

We haven't done much research into the problem beyond this.  For now, we
all just turn KVM off while we continue development.

~Gregory