[RFC PATCH 00/18] nvme: In-kernel TLS support for TCP

Hannes Reinecke
2023-03-21T12:44:05Z

Hi all,

finally I've managed to put all things together and enable in-kernel
TLS support for NVMe-over-TCP.

The patchset is based on the TLS upcall mechanism from Chuck Lever
(cf '[PATCH v7 0/2] Another crack at a handshake upcall mechanism'
posted to the linux netdev list), and requires the 'tlshd' userspace
daemon (https://github.com/oracle/ktls-utils) for the actual TLS handshake.

Theory of operation:
A dedicated '.nvme' keyring is created to hold the pre-shared keys (PSKs)
for the TLS handshake. Keys will have to be provisioned before TLS handshake
is attempted; I'll be sending a patch for nvme-cli separately.
After connection to the remote TCP port the client side will check if there are
matching PSKs, and call the TLS userspace daemon to initiate a TLS handshake.
The server side will be doing a MSG_PEEK on the first incoming PDU after
accept(), and check if it's an TCP ICREQ packet. If not it's assumed to be
a TLS ClientHello and the TLS userspace daemon is invoked for the TLS
handshake.
If the TLS handshake succeeds the userspace daemon will be activating
kTLS on the socket, and control is passed back to the kernel.

The one issue of note is the multiple identity handling.
The NVMe TCP spec defines up to 4 PSK identities, and
TLS 1.3 allows for several identities to be sent with the
ClientHello. So in theory we could sent the all in one go.
Sadly none of the userspace libraries implement this feature,
so we have to test each possible identity and terminate the
connection on failure.
With this patchset all possible identities need to be part of
the keyring, and the client side will be trying to establish
a TLS connection with each matching PSK from the keyring.
The beauty is that this method works without modification
to the existing nvme-cli; one only needs to provision PSKs
in the .nvme keyring and TLS handshake will be attempted.
As I'm not sure if that approach meets with general approval
I'm sending out this patchset as an RFC for now.

As usual, comments and reviews are welcome.

Hannes Reinecke (18):
nvme-keyring: register '.nvme' keyring
nvme-keyring: define a 'psk' keytype
nvme: add TCP TSAS definitions
nvme-tcp: add definitions for TLS cipher suites
nvme-tcp: implement recvmsg rx flow for TLS
nvme-tcp: call 'queue->data_ready()' in nvme_tcp_data_ready()
nvme/tcp: allocate socket file
nvme-tcp: enable TLS handshake upcall
nvme-tcp: add connect option 'tls'
nvme-tcp: fixup send workflow for kTLS
nvme-tcp: control message handling for recvmsg()
nvmet: make TCP sectype settable via configfs
nvmet-tcp: allocate socket file
security/keys: export key_lookup()
nvmet-tcp: enable TLS handshake upcall
nvmet-tcp: rework sendpage for kTLS
nvmet-tcp: control messages for recvmsg()
nvmet-tcp: peek icreq before starting TLS

drivers/nvme/common/Makefile | 2 +-
drivers/nvme/common/keyring.c | 132 ++++++++++
drivers/nvme/host/core.c | 10 +-
drivers/nvme/host/fabrics.c | 5 +
drivers/nvme/host/fabrics.h | 2 +
drivers/nvme/host/tcp.c | 450 +++++++++++++++++++++++++--------
drivers/nvme/target/configfs.c | 65 +++++
drivers/nvme/target/tcp.c | 407 ++++++++++++++++++++++++++---
include/linux/nvme-keyring.h | 20 ++
include/linux/nvme-tcp.h | 6 +
include/linux/nvme.h | 10 +
security/keys/key.c | 1 +
12 files changed, 965 insertions(+), 145 deletions(-)
create mode 100644 drivers/nvme/common/keyring.c
create mode 100644 include/linux/nvme-keyring.h

--
2.35.3