Re: [SPDK] SPDK + user space appliance

[Walker, Benjamin <benjamin.walker at intel.com>]

[-- Attachment #1: Type: text/plain, Size: 1264 bytes --]

On Sun, 2018-01-14 at 09:09 +0000, Shahar Salzman wrote:
> We have been integrating spdk into our system using a blockdev module,
> currently only a POC version.
> Our use case is a user space appliance processing IOs, with an SPDK frontend
> to do the NVMeF.
> 
> Currently all of the user bdevs are created via configuration file, but we are
> working to add functions + rpc's which allow creation/deletion of these
> namespaces.

Ok - so you have our NVMe-oF target in front and it routes I/Os to your system
via a custom bdev module that you wrote. This is exactly how we intended for you
to use it, so it looks good to me.

> IO is sent to user space via callback, implementation is up to user space, but
> obviously the longer it lingers there the lower the performance, we use a set
> of rings + threads processing them, so that the time spent in the appliance is
> minimal.
> Going back from user space we use a single ring (multiple producers single
> consumer) onto which the completions are inserted, and the ring poll function
> is registered with spdk core (spdk_poller_register).

What do you mean by I/O is sent to user space via callback? When was the I/O
ever in kernel space? Is your storage system running in the kernel?