From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
To: qemu-devel@nongnu.org, "Peter Maydell" <peter.maydell@linaro.org>,
"Gregory Price" <gregory.price@memverge.com>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Sajjan Rao" <sajjanr@gmail.com>,
"Dimitrios Palyvos" <dimitrios.palyvos@zptcorp.com>,
richard.henderson@linaro.org,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Eduardo Habkost" <eduardo@habkost.net>,
"Philippe Mathieu-Daudé" <philmd@linaro.org>
Cc: <linuxarm@huawei.com>, <linux-cxl@vger.kernel.org>
Subject: [PATCH v2 0/3 qemu] tcg/i386: Page tables in MMIO memory fixes (CXL)
Date: Mon, 19 Feb 2024 17:31:50 +0000 [thread overview]
Message-ID: <20240219173153.12114-1-Jonathan.Cameron@huawei.com> (raw)
v2: Changes documented in patch 3.
- I have not addressed Richard's comment on recursive locks as that
seems to be a more general issue not specific to this patch set.
CXL memory is interleaved at granularities as fine as 64 bytes.
To emulate this each read and write access undergoes address translation
similar to that used in physical hardware. This is done using
cfmws_ops for a memory region per CXL Fixed Memory Window (the PA address
range in the host that is interleaved across host bridges and beyond.
The OS programs interleaved decoders in the CXL Root Bridges, switch
upstream ports and the corresponding decoders CXL type 3 devices who
have to know the Host PA to Device PA mappings).
Unfortunately this CXL memory may be used as normal memory and anything
that can end up in RAM can be placed within it. As Linux has become
more capable of handling this memory we've started to get quite a few
bug reports for the QEMU support. However terrible the performance is
people seem to like running actual software stacks on it :(
This doesn't work for KVM - so for now CXL emulation remains TCG only.
(unless you are very careful on how it is used!) I plan to add some
safety guards at a later date to make it slightly harder for people
to shoot themselves in the foot + a more limited set of CXL functionality
that is safe (no interleaving!)
Previously we had some issues with TCG reading instructions from CXL
memory but that is now all working. This time the issues are around
the Page Tables being in the CXL memory + DMA buffers being placed in it.
The test setup I've been using is simple 2 way interleave via 2 root
ports below a single CXL root complex. After configuration in Linux
these are mapped to their own Numa Node and
numactl --membind=1 ls
followed by powering down the machine is sufficient to hit all the bugs
addressed in this series.
Thanks to Gregory, Peter and Alex for their help figuring this lot
out. Note that I've included one patch from Peter without a SoB
because so far it has only be posted in the discussion thread.
Whilst thread started back at:
https://lore.kernel.org/all/CAAg4PaqsGZvkDk_=PH+Oz-yeEUVcVsrumncAgegRKuxe_YoFhA@mail.gmail.com/
The QEMU part is from.
https://lore.kernel.org/all/20240201130438.00001384@Huawei.com/
Gregory Price (1):
target/i386: Enable page walking from MMIO memory
Jonathan Cameron (1):
tcg: Avoid double lock if page tables happen to be in mmio memory.
Peter Maydell (1):
accel/tcg: Set can_do_io at at start of lookup_tb_ptr helper
accel/tcg/cpu-exec.c | 8 ++++
accel/tcg/cputlb.c | 36 ++++++++++++++----
target/i386/tcg/sysemu/excp_helper.c | 57 +++++++++++++++-------------
3 files changed, 66 insertions(+), 35 deletions(-)
--
2.39.2
next reply other threads:[~2024-02-19 17:31 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-19 17:31 Jonathan Cameron [this message]
2024-02-19 17:31 ` [PATCH v2 1/3] accel/tcg: Set can_do_io at at start of lookup_tb_ptr helper Jonathan Cameron
2024-02-19 17:31 ` [PATCH v2 2/3] target/i386: Enable page walking from MMIO memory Jonathan Cameron
2024-02-19 17:31 ` [PATCH v2 3/3] tcg: Avoid double lock if page tables happen to be in mmio memory Jonathan Cameron
2024-02-22 17:35 ` Peter Maydell
2024-02-22 19:07 ` [PATCH v2 0/3 qemu] tcg/i386: Page tables in MMIO memory fixes (CXL) Richard Henderson
2024-03-06 15:04 ` Jonathan Cameron
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240219173153.12114-1-Jonathan.Cameron@huawei.com \
--to=jonathan.cameron@huawei.com \
--cc=alex.bennee@linaro.org \
--cc=dimitrios.palyvos@zptcorp.com \
--cc=eduardo@habkost.net \
--cc=gregory.price@memverge.com \
--cc=linux-cxl@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=pbonzini@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=philmd@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=richard.henderson@linaro.org \
--cc=sajjanr@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox