Re: [PATCH v2 0/3 qemu] tcg/i386: Page tables in MMIO memory fixes (CXL)

[Richard Henderson <richard.henderson@linaro.org>]

On 2/19/24 07:31, Jonathan Cameron wrote:
> v2: Changes documented in patch 3.
>     - I have not addressed Richard's comment on recursive locks as that
>       seems to be a more general issue not specific to this patch set.
> 
> CXL memory is interleaved at granularities as fine as 64 bytes.
> To emulate this each read and write access undergoes address translation
> similar to that used in physical hardware. This is done using
> cfmws_ops for a memory region per CXL Fixed Memory Window (the PA address
> range in the host that is interleaved across host bridges and beyond.
> The OS programs interleaved decoders in the CXL Root Bridges, switch
> upstream ports and the corresponding decoders CXL type 3 devices who
> have to know the Host PA to Device PA mappings).
> 
> Unfortunately this CXL memory may be used as normal memory and anything
> that can end up in RAM can be placed within it. As Linux has become
> more capable of handling this memory we've started to get quite a few
> bug reports for the QEMU support. However terrible the performance is
> people seem to like running actual software stacks on it :(
> 
> This doesn't work for KVM - so for now CXL emulation remains TCG only.
> (unless you are very careful on how it is used!)  I plan to add some
> safety guards at a later date to make it slightly harder for people
> to shoot themselves in the foot + a more limited set of CXL functionality
> that is safe (no interleaving!)
> 
> Previously we had some issues with TCG reading instructions from CXL
> memory but that is now all working. This time the issues are around
> the Page Tables being in the CXL memory + DMA buffers being placed in it.
> 
> The test setup I've been using is simple 2 way interleave via 2 root
> ports below a single CXL root complex.  After configuration in Linux
> these are mapped to their own Numa Node and
> numactl --membind=1 ls
> followed by powering down the machine is sufficient to hit all the bugs
> addressed in this series.
> 
> Thanks to Gregory, Peter and Alex for their help figuring this lot
> out. Note that I've included one patch from Peter without a SoB
> because so far it has only be posted in the discussion thread.
> 
> Whilst thread started back at:
> https://lore.kernel.org/all/CAAg4PaqsGZvkDk_=PH+Oz-yeEUVcVsrumncAgegRKuxe_YoFhA@mail.gmail.com/
> The QEMU part is from.
> https://lore.kernel.org/all/20240201130438.00001384@Huawei.com/
> 
> 
> Gregory Price (1):
>    target/i386: Enable page walking from MMIO memory
> 
> Jonathan Cameron (1):
>    tcg: Avoid double lock if page tables happen to be in mmio memory.
> 
> Peter Maydell (1):
>    accel/tcg: Set can_do_io at at start of lookup_tb_ptr helper

Thanks.  Queued patches 1 and 3 to tcg-next, and adjusted patch 3 to use BQL_LOCK_GUARD as 
suggested by Alex.


r~