On Mon, 2016-08-01 at 16:50 -0700, txcy uio wrote:
> Hello,
> 
> While submitting a readv command (spdk_nvme_ns_cmd_readv) I see the following error:
> 
> READ sqid:3 cid:127 nsid:1 lba:100 len:64
> INVALID_FIELD (00/02) sqid:3 cid:127 cdw0:0 sqhd:0000 p:0 m:0 dnr:1
> 
> 
> Based on this error what can I infer about the command that went wrong?

The SPDK NVMe driver is zero-copy, meaning we program the hardware to transfer data directly from
the buffers you specify. That's why all of the buffers need to be allocated from pinned memory (in
practice, that usually means they need to be allocated using rte_malloc from DPDK). NVMe devices
also have physical alignment restrictions. For instance, version 1.0 of the specification requires
us to program the hardware using a "physical resource page list" (PRP list), which consists of a
list of 4k physical memory pages. The first page does not have to be 4k aligned, but all others do,
and all but the first and last page must be 4k in length. Version 1.2 of the NVMe specification
optionally allows fully flexible scatter gather lists, but almost no commercially available devices
support scatter gather lists yet.

spdk_nvme_ns_cmd_readv is our interface for using the NVMe 1.2 scatter gather list capabilities. If your device doesn't support that, it's going to fail with an error like you are seeing. I now see that the purpose of spdk_nvme_ns_cmd_readv isn't clearly specified in the documentation (http://www.spdk.io/spdk/doc/nvme_8h.html#acb47ce7de6b6e963ec9fb8de261466ae) and if you aren't intimately familiar with the NVMe specification I can definitely see how you wouldn't realize that scatter gather lists require special hardware support. The documentation needs to be updated.

> 
> --Tyc
> 
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk