From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============5427490659069754376==" MIME-Version: 1.0 From: Kirubakaran Kaliannan Subject: Re: [SPDK] using nvmf target/initiator in spdk Date: Tue, 09 Aug 2016 11:26:53 +0530 Message-ID: <149d3d0c6423c829f50a4cd48b3ea8c4@mail.gmail.com> In-Reply-To: 1470153970.2387.248.camel@intel.com List-ID: To: spdk@lists.01.org --===============5427490659069754376== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Hi, 1) After following the steps in the document to configure the SPDK, I verify it by running the command /examples/ioat/verify/verify, and I get the following message. Is this expected ? ------------------------------------------------------------------------- EAL: Requesting 512 pages of size 2MB from socket 0 EAL: Requesting 512 pages of size 2MB from socket 1 EAL: TSC frequency is ~2400080 KHz EAL: Master lcore 0 is ready (tid=3D2fc3c900;cpuset=3D[0]) User configuration: Run time: 10 seconds Core mask: 0x1 Queue depth: 32 Not enough ioat channels found. Check that ioatdma driver is unloaded. <=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D # lsmod | grep ioat ioatdma 52511 0 dca 15130 1 ioatdma ------------------------------------------------------------------------- 2) Starting the nvmf_tgt using the below config ------------------------------------------------------------------------- [Subsystem1] NQN nqn.2016-06.io.spdk:cnode1 Mode Direct Listen RDMA 150.50.3.22 NVMe 0000:05:00.0 ------------------------------------------------------------------------- I get the following EGAIN continuously from the strace # strace app/nvmf_tgt/nvmf_tgt -c /etc/spdk/nvmf.conf -------------------------------------------------------------------------- open("/dev/infiniband/rdma_cm", O_RDWR) =3D 26 fcntl(26, F_SETFL, O_RDONLY|O_NONBLOCK) =3D 0 write(26, "\0\0\0\0\30\0\4\0`\22h\2\0\0\0\0000\5M9\377\177\0\0\6\1\2\0\0\0\0\0", 32) = =3D 32 write(26, "\2\0\0\0(\0\0\0\0\0\0\0\0\0\0\0\2\0\21D\0\0\0\0\0\0\0\0\0\0\0\0"..., 48) = =3D 48 write(26, "\5\0\0\0\20\0\310\0@\4M9\377\177\0\0\0\0\0\0\0\0\0\0", 24) =3D 24 write(26, "\7\0\0\0\10\0\0\0\0\0\0\0\n\0\0\0", 16) =3D 16 write(26, "\5\0\0\0\20\0\310\0\200\4M9\377\177\0\0\0\0\0\0\0\0\0\0", 24) = =3D 24 write(2, "*** NVMf Target Listening on por"..., 43*** NVMf Target Listening on port 4420 *** ) =3D 43 sendto(4, "<29>Aug 9 08:33:58 nvmf[10672]:"..., 76, MSG_NOSIGNAL, NULL, 0) =3D 76 write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) =3D -1 EAGAIN (Resour= ce temporarily unavailable) write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) =3D -1 EAGAIN (Resour= ce temporarily unavailable) write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) =3D -1 EAGAIN (Resour= ce temporarily unavailable) ..... ---------------------------------------------------------------------------- # Uname -a Linux server-1 3.18.19 #1 SMP PREEMPT Tue Feb 9 21:42:49 IST 2016 x86_64 x86_64 x86_64 GNU/Linux 3) The host time out discovering the nvme devices. Question: 1. How do I verify whether the SPDK is configured properly ? 2. From the above, is there anything I am missing in the configuration ? 3. Do we have to be at the Higher linux version than 3.18 to support SPDK ? Thanks -kiru -----Original Message----- From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, Benjam= in Sent: Tuesday, August 02, 2016 9:36 PM To: spdk(a)lists.01.org Subject: Re: [SPDK] using nvmf target/initiator in spdk On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote: > > > Hi, > > Do we have a document on, how to use the spdk to perform the following > > 1. List the devices that SPDK can see You can run the identify or nvme_manage example to get a nice list of all of the devices that SPDK can attach to. They are in examples/nvme. > 2. Start nvmf target driver with nvme devices We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html, which is accurate for our last tagged release but has gone stale for the code on 'master'. We'll have an updated one shortly. Right now it's fairly early stages, so you mostly have to just read the code. Our configuration file is reasonably well documented too. > 3. What nvmf initiator that we can use to connect this on the host > machine (ubuntu)? We test against the NVMe-oF initiator that will be added to the Linux kernel. It currently exists as a set of patches against kernel 4.8. Here is an article from Mellanox explaining how to set up your Linux kernel for NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against essentially any modern Linux kernel, so you only have to put kernel 4.8 on the initiator system. > > > Thanks, > -kiru > > _______________________________________________ > SPDK mailing list > SPDK(a)lists.01.org > https://lists.01.org/mailman/listinfo/spdk _______________________________________________ SPDK mailing list SPDK(a)lists.01.org https://lists.01.org/mailman/listinfo/spdk --===============5427490659069754376==--