Is idenditfy ctrl cmd supposed to return a 'result' for RDMA?

Is idenditfy ctrl cmd supposed to return a 'result' for RDMA?

[Daniel Wagner]

A bug was reported against nvme-cli that a setup works fine with
nvme-tcp but not with nvme-rdma[1]. The only difference I could spot
from user space PoV, is that RDMA returns 'funky' result values

# nvme id-ctrl /dev/nvme2 -vvv
opcode       : 06
flags        : 00
rsvd1        : 0000
nsid         : 00000000
cdw2         : 00000000
cdw3         : 00000000
data_len     : 00001000
metadata_len : 00000000
addr         : 3762f000
metadata     : 0
cdw10        : 00000001
cdw11        : 00000000
cdw12        : 00000000
cdw13        : 00000000
cdw14        : 00000000
cdw15        : 00000000
timeout_ms   : 00000000
result       : 3f3b0000
err          : 0
latency      : 7076 us

while for the same command, TCP returns 0.

I've tried to figure out where the source these values are

[ 4040.869872] nvmet: nvmet_execute_identify_ctrl:358
[ 4040.874002] nvme_submit_user_cmd:196 ffff88813f3b0000
[ 4040.875759] nvme_user_cmd:326 status 0 result ffff88813f3b0000

but got a bit lost in the RDMA code. So I am not sure, is this supposed to
happen?

[1] https://github.com/linux-nvme/libnvme/issues/807

Re: Is idenditfy ctrl cmd supposed to return a 'result' for RDMA?

[Daniel Wagner]

On Thu, Apr 11, 2024 at 12:51:40PM +0200, Daniel Wagner wrote:
> but got a bit lost in the RDMA code. So I am not sure, is this supposed to
> happen?

After reading up in the spec, it doesn't seem so. After adding


diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
index f5b7054a4a05..a297020f4bea 100644
--- a/drivers/nvme/target/admin-cmd.c
+++ b/drivers/nvme/target/admin-cmd.c
@@ -486,6 +486,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)

        kfree(id);
 out:
+       req->cqe->result.u64 = 0;
        nvmet_req_complete(req, status);
 }


the problem goes away. I've also noticed that many other commands return
a lot of 'funky' results as well. I haven't understood why setting
the result explicitly to 0 is necessary for RDMA yet.

Re: Is idenditfy ctrl cmd supposed to return a 'result' for RDMA?

[Christoph Hellwig]

On Thu, Apr 11, 2024 at 09:24:20PM +0200, Daniel Wagner wrote:
> On Thu, Apr 11, 2024 at 12:51:40PM +0200, Daniel Wagner wrote:
> > but got a bit lost in the RDMA code. So I am not sure, is this supposed to
> > happen?
> 
> After reading up in the spec, it doesn't seem so. After adding

I don't think anything requires reserved fields to be cleared, at least
I can't find anything in the spec.

> diff --git a/drivers/nvme/target/admin-cmd.c b/drivers/nvme/target/admin-cmd.c
> index f5b7054a4a05..a297020f4bea 100644
> --- a/drivers/nvme/target/admin-cmd.c
> +++ b/drivers/nvme/target/admin-cmd.c
> @@ -486,6 +486,7 @@ static void nvmet_execute_identify_ctrl(struct nvmet_req *req)
> 
>         kfree(id);
>  out:
> +       req->cqe->result.u64 = 0;
>         nvmet_req_complete(req, status);
>  }
> 
> 
> the problem goes away. I've also noticed that many other commands return
> a lot of 'funky' results as well. I haven't understood why setting
> the result explicitly to 0 is necessary for RDMA yet.

that being said I'm fine with clearing the CQE, but can we find a way
to do this in common code?