[SPDK] using nvmf target/initiator in spdk

[SPDK] using nvmf target/initiator in spdk

[Kirubakaran Kaliannan]

[-- Attachment #1: Type: text/plain, Size: 312 bytes --]

Hi,



Do we have a document on, how to use the spdk to perform the following



1.       List the devices that SPDK can see

2.       Start nvmf target driver with nvme devices

3.       What nvmf initiator that we can use to connect this on the host
machine (ubuntu)?





Thanks,

-kiru

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 2741 bytes --]

Re: [SPDK] using nvmf target/initiator in spdk

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 1451 bytes --]

On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote:
>  
>  
> Hi,
>  
> Do we have a document on, how to use the spdk to perform the following
>  
> 1.       List the devices that SPDK can see

You can run the identify or nvme_manage example to get a nice list of all of the devices that SPDK
can attach to. They are in examples/nvme.

> 2.       Start nvmf target driver with nvme devices

We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html, which is accurate for our
last tagged release but has gone stale for the code on 'master'. We'll have an updated one shortly.
Right now it's fairly early stages, so you mostly have to just read the code. Our configuration file
is reasonably well documented too.

> 3.       What nvmf initiator that we can use to connect this on the host machine (ubuntu)?

We test against the NVMe-oF initiator that will be added to the Linux kernel. It currently exists as
a set of patches against kernel 4.8. Here is an article from Mellanox explaining how to set up your
Linux kernel for NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against
essentially any modern Linux kernel, so you only have to put kernel 4.8 on the initiator system.

>  
>  
> Thanks,
> -kiru
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] using nvmf target/initiator in spdk

[Minfei Huang]

[-- Attachment #1: Type: text/plain, Size: 1741 bytes --]

Hi.

FYI

Thanks
Minfei

> On Aug 3, 2016, at 00:06, Walker, Benjamin <benjamin.walker(a)intel.com> wrote:
> 
> On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote:
>>  
>>  
>> Hi,
>>  
>> Do we have a document on, how to use the spdk to perform the following
>>  
>> 1.       List the devices that SPDK can see
> 
> You can run the identify or nvme_manage example to get a nice list of all of the devices that SPDK
> can attach to. They are in examples/nvme.
> 
>> 2.       Start nvmf target driver with nvme devices
> 
> We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html, which is accurate for our
> last tagged release but has gone stale for the code on 'master'. We'll have an updated one shortly.
> Right now it's fairly early stages, so you mostly have to just read the code. Our configuration file
> is reasonably well documented too.
> 
>> 3.       What nvmf initiator that we can use to connect this on the host machine (ubuntu)?
> 
> We test against the NVMe-oF initiator that will be added to the Linux kernel. It currently exists as
> a set of patches against kernel 4.8. Here is an article from Mellanox explaining how to set up your
> Linux kernel for NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against
> essentially any modern Linux kernel, so you only have to put kernel 4.8 on the initiator system.
> 
>>  
>>  
>> Thanks,
>> -kiru
>> 
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] using nvmf target/initiator in spdk

[Kirubakaran Kaliannan]

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]

Thanks Walker,
I am able to get the nvmef_tgt started. I am working on porting/upgrading
the latest linux kernel (4.8) on my another host.
I will get back to you if I have any challenges in connecting to the nvme
target.

Thanks
-kiru

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, Benjamin
Sent: Tuesday, August 02, 2016 9:36 PM
To: spdk(a)lists.01.org
Subject: Re: [SPDK] using nvmf target/initiator in spdk

On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote:
>
>
> Hi,
>
> Do we have a document on, how to use the spdk to perform the following
>
> 1.       List the devices that SPDK can see

You can run the identify or nvme_manage example to get a nice list of all of
the devices that SPDK can attach to. They are in examples/nvme.

> 2.       Start nvmf target driver with nvme devices

We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html,
which is accurate for our last tagged release but has gone stale for the
code on 'master'. We'll have an updated one shortly.
Right now it's fairly early stages, so you mostly have to just read the
code. Our configuration file is reasonably well documented too.

> 3.       What nvmf initiator that we can use to connect this on the host
> machine (ubuntu)?

We test against the NVMe-oF initiator that will be added to the Linux
kernel. It currently exists as a set of patches against kernel 4.8. Here is
an article from Mellanox explaining how to set up your Linux kernel for
NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against
essentially any modern Linux kernel, so you only have to put kernel 4.8 on
the initiator system.

>
>
> Thanks,
> -kiru
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] using nvmf target/initiator in spdk

[Kirubakaran Kaliannan]

[-- Attachment #1: Type: text/plain, Size: 5230 bytes --]

Hi,

I have the following EAGAIN issue, when I start the nvmf_tgt on the server.

Do we have any OFED dependencies or any other package/configuration ? I
followed the nvmf_getting started document.

# tail -f /etc/spdk/nvmf.conf
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Mode Direct
  Listen RDMA 150.50.3.22:4420
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:05:00.0

# strace app/nvmf_tgt/nvmf_tgt -c /etc/spdk/nvmf.conf > /tmp/out 2>&1

open("/dev/infiniband/rdma_cm", O_RDWR) = 26
write(2, "*** NVMf Target Listening on por"..., 43*** NVMf Target Listening
on port 4420 ***
) = 43
sendto(4, "<29>Aug  4 10:22:41 nvmf[11287]:"..., 76, MSG_NOSIGNAL, NULL, 0)
= 76
write(26, "\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1 EAGAIN
(Resource temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1 EAGAIN
(Resource temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1 EAGAIN
(Resource temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1 EAGAIN
(Resource temporarily unavailable)
......

# dpkg -l | grep -i ofed
ii  ibverbs-utils                           1.1.8mlnx1-OFED.2.4.1.0.7
amd64        Examples for the libibverbs library
ii  libibverbs1                             1.1.8mlnx1-OFED.2.4.1.0.7
amd64        Library for direct userspace use of RDMA (InfiniBand/iWARP)
ii  libmlx4-1                               1.0.6mlnx1-OFED.2.4.0.1.2
amd64        Userspace driver for Mellanox ConnectX InfiniBand HCAs
ii  librdmacm-utils                         1.0.19mlnx-OFED.2.4.28.g50d12b8
amd64        Examples for the librdmacm library
ii  librdmacm1                              1.0.19mlnx-OFED.2.4.28.g50d12b8
amd64        Userspace RDMA Connection Manager
ii  mlnx-ofed-kernel-utils                  2.4-OFED.2.4.1.1.2.1.g5b97871
amd64        Userspace tools to restart and tune mlnx-ofed kernel modules
ii  ofed-scripts                            2.4-OFED.2.4.1.1.2
amd64        MLNX_OFED utilities

# nvme_manage list-controller

NVMe Controller:        0000:05:00.00
============================
Controller Capabilities/Features
Controller ID:          0
Serial Number:          CVMD439300582P0EGN

Admin Command Set Attributes
============================
Namespace Manage And Attach:            Not Supported
Namespace Format:                       Supported

NVM Command Set Attributes
============================
Namespace format operation applies to all namespaces

Namespace Attributes
============================
Namespace ID:1
Size (in LBAs):              3907029168 (3726M)
Capacity (in LBAs):          3907029168 (3726M)
Utilization (in LBAs):       3907029168 (3726M)
Format Progress Indicator:   Not Supported
Number of LBA Formats:       7
Current LBA Format:          LBA Format #00
LBA Format #00: Data Size:   512  Metadata Size:     0
LBA Format #01: Data Size:   512  Metadata Size:     8
LBA Format #02: Data Size:   512  Metadata Size:    16
LBA Format #03: Data Size:  4096  Metadata Size:     0
LBA Format #04: Data Size:  4096  Metadata Size:     8
LBA Format #05: Data Size:  4096  Metadata Size:    64
LBA Format #06: Data Size:  4096  Metadata Size:   128
Data Protection Capabilities:PIT1 Location: Tail
Data Protection Setting:     N/A
Multipath IO and Sharing:    Not Supported

Thanks
-kiru

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, Benjamin
Sent: Tuesday, August 02, 2016 9:36 PM
To: spdk(a)lists.01.org
Subject: Re: [SPDK] using nvmf target/initiator in spdk

On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote:
>
>
> Hi,
>
> Do we have a document on, how to use the spdk to perform the following
>
> 1.       List the devices that SPDK can see

You can run the identify or nvme_manage example to get a nice list of all of
the devices that SPDK can attach to. They are in examples/nvme.

> 2.       Start nvmf target driver with nvme devices

We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html,
which is accurate for our last tagged release but has gone stale for the
code on 'master'. We'll have an updated one shortly.
Right now it's fairly early stages, so you mostly have to just read the
code. Our configuration file is reasonably well documented too.

> 3.       What nvmf initiator that we can use to connect this on the host
> machine (ubuntu)?

We test against the NVMe-oF initiator that will be added to the Linux
kernel. It currently exists as a set of patches against kernel 4.8. Here is
an article from Mellanox explaining how to set up your Linux kernel for
NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against
essentially any modern Linux kernel, so you only have to put kernel 4.8 on
the initiator system.

>
>
> Thanks,
> -kiru
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] using nvmf target/initiator in spdk

[Kirubakaran Kaliannan]

[-- Attachment #1: Type: text/plain, Size: 6334 bytes --]

Hi All,

I may need your help to resolve this issue (with spdk (3.18 kernel) vs
nvme-fabrics  (4.17+rc2 kernel)

I have the nvme controller from Intel on server1, and exported using
nvmf_tgt.

On server 2, I have the latest kernel with nvme-fabrics installed. Trying to
run discover, and I am hitting the following error. (using the native
rdma/mlx drivers)

# nvme discover -t rdma -a 150.50.3.22 -s 4420
Aug  5 14:05:50.202036 server-0000027c-vc-1 kernel: [ 6071.642654] nvme
nvme0: Connect command failed, error wo/DNR bit: 6 (NVME_SC_INTERNAL)


On server1, strace of nvmf_tgt, I am getting following error continuously

write(26, "\f\0\0\0\10\0H\1 hRT\375\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1 hRT\375\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)



Thanks,
-kiru

-----Original Message-----
From: Kirubakaran Kaliannan [mailto:kirubak(a)zadarastorage.com]
Sent: Thursday, August 04, 2016 1:08 PM
To: 'Storage Performance Development Kit'
Subject: RE: [SPDK] using nvmf target/initiator in spdk



Hi,

I have the following EAGAIN issue, when I start the nvmf_tgt on the server.

Do we have any OFED dependencies or any other package/configuration ? I
followed the nvmf_getting started document.

# tail -f /etc/spdk/nvmf.conf
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Mode Direct
  Listen RDMA 150.50.3.22:4420
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:05:00.0

# strace app/nvmf_tgt/nvmf_tgt -c /etc/spdk/nvmf.conf > /tmp/out 2>&1

open("/dev/infiniband/rdma_cm", O_RDWR) = 26 write(2, "*** NVMf Target
Listening on por"..., 43*** NVMf Target Listening on port 4420 ***
) = 43
sendto(4, "<29>Aug  4 10:22:41 nvmf[11287]:"..., 76, MSG_NOSIGNAL, NULL, 0)
= 76 write(26, "\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1
EAGAIN (Resource temporarily unavailable) write(26,
"\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable) write(26,
"\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable) write(26,
"\f\0\0\0\10\0H\1\220\335\241\236\374\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable) ......

# dpkg -l | grep -i ofed
ii  ibverbs-utils                           1.1.8mlnx1-OFED.2.4.1.0.7
amd64        Examples for the libibverbs library
ii  libibverbs1                             1.1.8mlnx1-OFED.2.4.1.0.7
amd64        Library for direct userspace use of RDMA (InfiniBand/iWARP)
ii  libmlx4-1                               1.0.6mlnx1-OFED.2.4.0.1.2
amd64        Userspace driver for Mellanox ConnectX InfiniBand HCAs
ii  librdmacm-utils                         1.0.19mlnx-OFED.2.4.28.g50d12b8
amd64        Examples for the librdmacm library
ii  librdmacm1                              1.0.19mlnx-OFED.2.4.28.g50d12b8
amd64        Userspace RDMA Connection Manager
ii  mlnx-ofed-kernel-utils                  2.4-OFED.2.4.1.1.2.1.g5b97871
amd64        Userspace tools to restart and tune mlnx-ofed kernel modules
ii  ofed-scripts                            2.4-OFED.2.4.1.1.2
amd64        MLNX_OFED utilities

# nvme_manage list-controller

NVMe Controller:        0000:05:00.00
============================
Controller Capabilities/Features
Controller ID:          0
Serial Number:          CVMD439300582P0EGN

Admin Command Set Attributes
============================
Namespace Manage And Attach:            Not Supported
Namespace Format:                       Supported

NVM Command Set Attributes
============================
Namespace format operation applies to all namespaces

Namespace Attributes
============================
Namespace ID:1
Size (in LBAs):              3907029168 (3726M)
Capacity (in LBAs):          3907029168 (3726M)
Utilization (in LBAs):       3907029168 (3726M)
Format Progress Indicator:   Not Supported
Number of LBA Formats:       7
Current LBA Format:          LBA Format #00
LBA Format #00: Data Size:   512  Metadata Size:     0
LBA Format #01: Data Size:   512  Metadata Size:     8
LBA Format #02: Data Size:   512  Metadata Size:    16
LBA Format #03: Data Size:  4096  Metadata Size:     0
LBA Format #04: Data Size:  4096  Metadata Size:     8
LBA Format #05: Data Size:  4096  Metadata Size:    64
LBA Format #06: Data Size:  4096  Metadata Size:   128
Data Protection Capabilities:PIT1 Location: Tail
Data Protection Setting:     N/A
Multipath IO and Sharing:    Not Supported

Thanks
-kiru

-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, Benjamin
Sent: Tuesday, August 02, 2016 9:36 PM
To: spdk(a)lists.01.org
Subject: Re: [SPDK] using nvmf target/initiator in spdk

On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote:
>
>
> Hi,
>
> Do we have a document on, how to use the spdk to perform the following
>
> 1.       List the devices that SPDK can see

You can run the identify or nvme_manage example to get a nice list of all of
the devices that SPDK can attach to. They are in examples/nvme.

> 2.       Start nvmf target driver with nvme devices

We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html,
which is accurate for our last tagged release but has gone stale for the
code on 'master'. We'll have an updated one shortly.
Right now it's fairly early stages, so you mostly have to just read the
code. Our configuration file is reasonably well documented too.

> 3.       What nvmf initiator that we can use to connect this on the host
> machine (ubuntu)?

We test against the NVMe-oF initiator that will be added to the Linux
kernel. It currently exists as a set of patches against kernel 4.8. Here is
an article from Mellanox explaining how to set up your Linux kernel for
NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against
essentially any modern Linux kernel, so you only have to put kernel 4.8 on
the initiator system.

>
>
> Thanks,
> -kiru
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] using nvmf target/initiator in spdk

[Kirubakaran Kaliannan]

[-- Attachment #1: Type: text/plain, Size: 4607 bytes --]

Hi,

1)
After following the steps in the document to configure the SPDK, I verify it
by running the command
/examples/ioat/verify/verify, and I get the following message. Is this
expected ?

-------------------------------------------------------------------------
EAL: Requesting 512 pages of size 2MB from socket 0
EAL: Requesting 512 pages of size 2MB from socket 1
EAL: TSC frequency is ~2400080 KHz
EAL: Master lcore 0 is ready (tid=2fc3c900;cpuset=[0])
User configuration:
Run time:       10 seconds
Core mask:      0x1
Queue depth:    32

Not enough ioat channels found. Check that ioatdma driver is unloaded.
<================

# lsmod | grep ioat
ioatdma                52511  0
dca                    15130  1 ioatdma
-------------------------------------------------------------------------

2)
Starting the nvmf_tgt using the below config

-------------------------------------------------------------------------
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Mode Direct
  Listen RDMA 150.50.3.22
  NVMe 0000:05:00.0
-------------------------------------------------------------------------

I get the following EGAIN continuously from the strace

# strace app/nvmf_tgt/nvmf_tgt -c /etc/spdk/nvmf.conf

--------------------------------------------------------------------------
open("/dev/infiniband/rdma_cm", O_RDWR) = 26
fcntl(26, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
write(26,
"\0\0\0\0\30\0\4\0`\22h\2\0\0\0\0000\5M9\377\177\0\0\6\1\2\0\0\0\0\0", 32) =
32
write(26,
"\2\0\0\0(\0\0\0\0\0\0\0\0\0\0\0\2\0\21D\0\0\0\0\0\0\0\0\0\0\0\0"..., 48) =
48
write(26, "\5\0\0\0\20\0\310\0@\4M9\377\177\0\0\0\0\0\0\0\0\0\0", 24) = 24
write(26, "\7\0\0\0\10\0\0\0\0\0\0\0\n\0\0\0", 16) = 16
write(26, "\5\0\0\0\20\0\310\0\200\4M9\377\177\0\0\0\0\0\0\0\0\0\0", 24) =
24
write(2, "*** NVMf Target Listening on por"..., 43*** NVMf Target Listening
on port 4420 ***
) = 43
sendto(4, "<29>Aug  9 08:33:58 nvmf[10672]:"..., 76, MSG_NOSIGNAL, NULL, 0)
= 76
write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)
write(26, "\f\0\0\0\10\0H\1\320\3M9\377\177\0\0", 16) = -1 EAGAIN (Resource
temporarily unavailable)
.....
----------------------------------------------------------------------------

# Uname -a
Linux server-1 3.18.19 #1 SMP PREEMPT Tue Feb 9 21:42:49 IST 2016 x86_64
x86_64 x86_64 GNU/Linux

3)
The host time out discovering the nvme devices.

Question:

1. How do I verify whether the SPDK is configured properly ?
2. From the above, is there anything I am missing in the configuration ?
3. Do we have to be at the Higher linux version than 3.18 to support SPDK ?


Thanks
-kiru


-----Original Message-----
From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Walker, Benjamin
Sent: Tuesday, August 02, 2016 9:36 PM
To: spdk(a)lists.01.org
Subject: Re: [SPDK] using nvmf target/initiator in spdk

On Tue, 2016-08-02 at 15:52 +0530, Kirubakaran Kaliannan wrote:
>
>
> Hi,
>
> Do we have a document on, how to use the spdk to perform the following
>
> 1.       List the devices that SPDK can see

You can run the identify or nvme_manage example to get a nice list of all of
the devices that SPDK can attach to. They are in examples/nvme.

> 2.       Start nvmf target driver with nvme devices

We have this page: http://www.spdk.io/spdk/doc/nvmf_getting_started.html,
which is accurate for our last tagged release but has gone stale for the
code on 'master'. We'll have an updated one shortly.
Right now it's fairly early stages, so you mostly have to just read the
code. Our configuration file is reasonably well documented too.

> 3.       What nvmf initiator that we can use to connect this on the host
> machine (ubuntu)?

We test against the NVMe-oF initiator that will be added to the Linux
kernel. It currently exists as a set of patches against kernel 4.8. Here is
an article from Mellanox explaining how to set up your Linux kernel for
NVMe-oF: https://community.mellanox.com/docs/DOC-2508. SPDK can run against
essentially any modern Linux kernel, so you only have to put kernel 4.8 on
the initiator system.

>
>
> Thanks,
> -kiru
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] using nvmf target/initiator in spdk

[Verkamp, Daniel]

[-- Attachment #1: Type: text/plain, Size: 1666 bytes --]

Hi,

Splitting off just the I/OAT question (see below).

On Tue, 2016-08-09 at 11:26 +0530, Kirubakaran Kaliannan wrote:
> Hi,
> 
> 1)
> After following the steps in the document to configure the SPDK, I
> verify it
> by running the command
> /examples/ioat/verify/verify, and I get the following message. Is this
> expected ?
> 
> ----------------------------------------------------------------------
> ---
> EAL: Requesting 512 pages of size 2MB from socket 0
> EAL: Requesting 512 pages of size 2MB from socket 1
> EAL: TSC frequency is ~2400080 KHz
> EAL: Master lcore 0 is ready (tid=2fc3c900;cpuset=[0])
> User configuration:
> Run time:       10 seconds
> Core mask:      0x1
> Queue depth:    32
> 
> Not enough ioat channels found. Check that ioatdma driver is unloaded.
> <================
> 
> # lsmod | grep ioat
> ioatdma                52511  0
> dca                    15130  1 ioatdma

What CPU/chipset are you using for this test?

Can you verify that you see some I/OAT devices in lspci? If you use
`lspci -vn`, they should show up something like:

00:04.0 0880: 8086:2f20 (rev 02)
[...]
        Kernel driver in use: uio_pci_generic
        Kernel modules: ioatdma

You can find them based on the ioatdma module or by checking against the
list in lib/ioat/ioat_pci.h.

Any DMA channels you want to use must be unbound from the Linux kernel
ioatdma driver, either by totally unloading it (rmmod ioatdma) or by
using something like scripts/setup.sh to selectively change the driver
for specific devices.

Thanks,
-- Daniel