Re: [SPDK] using nvmf target/initiator in spdk

[Kirubakaran Kaliannan <kirubak at zadarastorage.com>]

[-- Attachment #1: Type: text/plain, Size: 4607 bytes --]

Hi,

1)
After following the steps in the document to configure the SPDK, I verify it
by running the command
/examples/ioat/verify/verify, and I get the following message. Is this
expected ?

-------------------------------------------------------------------------
EAL: Requesting 512 pages of size 2MB from socket 0
EAL: Requesting 512 pages of size 2MB from socket 1
EAL: TSC frequency is ~2400080 KHz
EAL: Master lcore 0 is ready (tid=2fc3c900;cpuset=[0])
User configuration:
Run time:       10 seconds
Core mask:      0x1
Queue depth:    32

Not enough ioat channels found. Check that ioatdma driver is unloaded.
<================

# lsmod | grep ioat
ioatdma                52511  0
dca                    15130  1 ioatdma
-------------------------------------------------------------------------

2)
Starting the nvmf_tgt using the below config

-------------------------------------------------------------------------
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Mode Direct
  Listen RDMA 150.50.3.22
  NVMe 0000:05:00.0
-------------------------------------------------------------------------

I get the following EGAIN continuously from the strace

# strace app/nvmf_tgt/nvmf_tgt -c /etc/spdk/nvmf.conf

--------------------------------------------------------------------------
open("/dev/infiniband/rdma_cm", O_RDWR) = 26
fcntl(26, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
write(26,
"\0\0\0\0\30\0\4\0`\22h\2\0\0\0\0000\5M9\377\177\0\0\6\1\2\0\0\0\0\0", 32) =
32
write(26,
"\2\0\0\0(\0\0\0\0\0\0\0\0\0\0\0\2\0\21D\0\0\0\0\0\0\0\0\0\0\0\0"..., 48) =
48
write(26, "\5\0\0\0\20\0\310\0@\4M9\377\177\0\0\0\0\0\0\0\0\0\0", 24) = 24
write(26, "\7\0\0\0\10\0\0\0\0\0\0\0\n\0\0\0", 16) = 16
write(26, "\5\0\0\0\20\0\310\0\200\4M9\377\177\0\0\0\0\0\0\0\0\0\0", 24) =
24
write(2, "*** NVMf Target Listening on por"..., 43*** NVMf Target Listening
on port 4420 ***
) = 43
sendto(4, "<29>Aug  9 08:33:58 nvmf[10672]:"..., 76, MSG_NOSIGNAL, NULL, 0)
= 76
write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)
.....
----------------------------------------------------------------------------

# Uname -a
Linux server-1 3.18.19 #1 SMP PREEMPT Tue Feb 9 21:42:49 IST 2016 x86_64
x86_64 x86_64 GNU/Linux

3)
The host time out discovering the nvme devices.

Question:

1. How do I verify whether the SPDK is configured properly ?
2. From the above, is there anything I am missing in the configuration ?
3. Do we have to be at the Higher linux version than 3.18 to support SPDK ?


Thanks
-kiru


-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, Benjamin
Sent: Tuesday, August 02, 2016 9:36 PM
To: spdk(a)lists.01.org
Subject: Re: [SPDK] using nvmf target/initiator in spdk

On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote:
>
>
> Hi,
>
> Do we have a document on, how to use the spdk to perform the following
>
> 1.       List the devices that SPDK can see

You can run the identify or nvme_manage example to get a nice list of all of
the devices that SPDK can attach to. They are in examples/nvme.

> 2.       Start nvmf target driver with nvme devices

We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html,
which is accurate for our last tagged release but has gone stale for the
code on 'master'. We'll have an updated one shortly.
Right now it's fairly early stages, so you mostly have to just read the
code. Our configuration file is reasonably well documented too.

> 3.       What nvmf initiator that we can use to connect this on the host
> machine (ubuntu)?

We test against the NVMe-oF initiator that will be added to the Linux
kernel. It currently exists as a set of patches against kernel 4.8. Here is
an article from Mellanox explaining how to set up your Linux kernel for
NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against
essentially any modern Linux kernel, so you only have to put kernel 4.8 on
the initiator system.

>
>
> Thanks,
> -kiru
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk