From mboxrd@z Thu Jan  1 00:00:00 1970
Content-Type: multipart/mixed; boundary="===============6588362562705397560=="
MIME-Version: 1.0
From: Walker, Benjamin <benjamin.walker at intel.com>
Subject: [SPDK] NVMe-oF and multiple targets
Date: Thu, 07 Nov 2019 18:25:36 +0000
Message-ID: <ffe4ffdd0269b84ba3723189dfe3b6a49e769abf.camel@intel.com>
List-ID: <spdk@lists.01.org>
To: spdk@lists.01.org

--===============6588362562705397560==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

Hi all,

I've got an open design question that I wanted to run by the community rega=
rding
the NVMe-oF target. There are a few primitives that the library currently
exposes:

1) NVMe-oF transports are a networking transport abstraction. These have me=
mory
and request object pools and can create and destroy connections.
2) NVMe-oF poll groups are sets of NVMe-oF queue pairs (that aren't necessa=
rily
related to one another). A single poll group can aggregate connections from
multiple different transports.
3) NVMe-oF subsystems are sets of related namespaces. A subsystem is effect=
ively
an access control list.
4) NVMe-oF controllers are network sessions. It's a set of NVMe-oF queue pa=
irs
(connections) that are all connected to the same subsystem and accessing the
same namespaces.
6) NVMe-oF targets are a set of NVMe-oF subsystems, controllers, and poll
groups. The target object really defines the NVMe-oF discovery service.

Historically, the SPDK NVMe-oF target application created a single, global =
NVMe-
oF target object. But we're trying to generalize the underlying nvmf librar=
y to
support multiple targets so that a single application can support multiple
discovery services for more complex use cases.

My question is around how to map transports (#1) to targets (#6). There are=
 two
paths we could go down. I'll use the RDMA transport as an example.

A) Each target gets its own transport. That means there would be two RDMA
transports for two targets, and each transport would allocate it's own pool=
 of
independent resources.
B) transports can be shared across targets. Two different targets can share=
 a
single RDMA transport. They can't share connections or listen on the same
addresses, of course, but they share the same request/buffer pools internal=
ly.

Does anyone have a strong opinion on what the right choice is here?

Thanks,
Ben

--===============6588362562705397560==--