Linux CXL
 help / color / mirror / Atom feed
From: Jonathan Cameron <Jonathan.Cameron@huawei.com>
To: <linux-cxl@vger.kernel.org>,
	Dan Williams <dan.j.williams@intel.com>,
	Ira Weiny <ira.weiny@intel.com>,
	Vishal Verma <vishal.l.verma@intel.com>,
	Alison Schofield <alison.schofield@intel.com>,
	Dave Jiang <dave.jiang@intel.com>,
	Davidlohr Bueso <dave@stgolabs.net>
Cc: <linuxarm@huawei.com>, Bjorn Helgaas <bhelgaas@google.com>
Subject: [PATCH 0/4] CXL: Standalone switch CCI driver
Date: Mon, 16 Oct 2023 13:53:19 +0100	[thread overview]
Message-ID: <20231016125323.18318-1-Jonathan.Cameron@huawei.com> (raw)

Note that the PCI SIG has recently release the MMPT specification which is
similar to the CXL mailboxes. I would suggest that we don't get too
distracted by that, but keep in mind that we may end up factoring some
aspects of this code out into the PCI core and just use them here (at some
future date!)

https://members.pcisig.com/wg/PCI-SIG/document/20109?uploaded=1

Based on v6.6-rc6

Chances since rfc4. 
- Mostly a rebase because of fixes around sanitize handling.
- Dropped new command support. For, now RAW all the way.
- Aim is to support getting the QEMU emulation upstream.

I'll be posting user space code to test this against the QEMU emulation shortly.
I just need to finish writing the documentation as getting it up and running
is less trivial than I'd like (lots of moving parts).
(I need this series on the list so I can point to it ;)

Introduction:

CXL r3.0 introduced the option for a PCI function, intended to sit on an
upstream port of a CXL switch.  This function provides a mailbox interface
similar to that seen on CXL type 3 devices. However, the command set is
mostly different and intended for Fabric management. Note however that as
we add support for multi headed devices (MHDs) a subset of commands will
be available on selected MHD type 3 mailboxes. (tunnelling for DCD commands
for example)

See: CXL rev 3.0
7.2.9 Switch Mailbox CCI
8.1.13 Switch Mailbox CCI Configuration Space Layout
8.2.8.6 Switch Mailbox CCI capability 

It is probably relatively unusual that a typical host of CXL devices will
have access to the one of these devices, in many cases they will be on a
port connected to a BMC or similar. There are a few use cases where the
host might be in charge of the configuration.

These are very convenient for testing in conjunction with the QEMU
emulation. CXL switch and type 3 emulation is in QEMU is not complex
enough to make these particular interesting but that should change soon.

For now don't provide any additional commands over those defined for the
main CXL mailbox and in practice that means the RAW command path under
CONFIG_CXL_MEM_RAW_COMMANDS is used with the IOCTL to interact with this
device and the fabric beyond it.

The foot guns are behind the same "don't enable this unless you know what
you are doing" barrier as they are for type 3 devices. Note that the blast
radius is far worse than for a CXL type 3 mailbox. This device can be used
to rip memory out from under _other live hosts_.

Jonathan Cameron (4):
  cxl: mbox: Preparatory move of functions to core/mbox.c and cxlmbox.h
  cxl: mbox: Factor out the mbox specific data for reuse in switch cci
  PCI: Add PCI_CLASS_SERIAL_CXL_SWITCH_CCI class ID to pci_ids.h
  cxl/pci: Add support for stand alone CXL Switch mailbox CCI

 drivers/cxl/Kconfig          |  14 ++
 drivers/cxl/Makefile         |   2 +
 drivers/cxl/core/Makefile    |   1 +
 drivers/cxl/core/core.h      |  12 +-
 drivers/cxl/core/mbox.c      | 453 +++++++++++++++++++++++++++--------
 drivers/cxl/core/memdev.c    |  44 ++--
 drivers/cxl/core/regs.c      |  35 ++-
 drivers/cxl/core/switchdev.c | 129 ++++++++++
 drivers/cxl/cxl.h            |   4 +-
 drivers/cxl/cxlmbox.h        | 201 ++++++++++++++++
 drivers/cxl/cxlmem.h         | 176 ++------------
 drivers/cxl/pci.c            | 442 ++++++++++------------------------
 drivers/cxl/pmem.c           |   6 +-
 drivers/cxl/security.c       |  13 +-
 drivers/cxl/switch.h         |  19 ++
 drivers/cxl/switchdev.c      | 169 +++++++++++++
 include/linux/pci_ids.h      |   1 +
 17 files changed, 1123 insertions(+), 598 deletions(-)
 create mode 100644 drivers/cxl/core/switchdev.c
 create mode 100644 drivers/cxl/cxlmbox.h
 create mode 100644 drivers/cxl/switch.h
 create mode 100644 drivers/cxl/switchdev.c

-- 
2.39.2


             reply	other threads:[~2023-10-16 12:53 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-10-16 12:53 Jonathan Cameron [this message]
2023-10-16 12:53 ` [PATCH 1/4] cxl: mbox: Preparatory move of functions to core/mbox.c and cxlmbox.h Jonathan Cameron
2023-11-13  0:09   ` Dan Williams
2023-11-13 19:39     ` Sumanesh Samanta
2023-11-28 17:26       ` Jonathan Cameron
2023-11-28 17:42     ` Jonathan Cameron
2023-10-16 12:53 ` [PATCH 2/4] cxl: mbox: Factor out the mbox specific data for reuse in switch cci Jonathan Cameron
2023-10-16 12:53 ` [PATCH 3/4] PCI: Add PCI_CLASS_SERIAL_CXL_SWITCH_CCI class ID to pci_ids.h Jonathan Cameron
2023-10-16 16:58   ` Bjorn Helgaas
2023-10-16 12:53 ` [PATCH 4/4] cxl/pci: Add support for stand alone CXL Switch mailbox CCI Jonathan Cameron
2023-11-16 14:08   ` Dan Williams
2023-10-16 15:17 ` [PATCH 0/4] CXL: Standalone switch CCI driver Jonathan Cameron

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20231016125323.18318-1-Jonathan.Cameron@huawei.com \
    --to=jonathan.cameron@huawei.com \
    --cc=alison.schofield@intel.com \
    --cc=bhelgaas@google.com \
    --cc=dan.j.williams@intel.com \
    --cc=dave.jiang@intel.com \
    --cc=dave@stgolabs.net \
    --cc=ira.weiny@intel.com \
    --cc=linux-cxl@vger.kernel.org \
    --cc=linuxarm@huawei.com \
    --cc=vishal.l.verma@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox