[Qemu-devel] [PATCH] nvme: Fix get/set number of queues feature, again

[Qemu-devel] [PATCH] nvme: Fix get/set number of queues feature, again

[Dan Aloni]

The number of queues that should be return by the admin command should:

  1) Only mention the number of non-admin queues.
  2) It is zero-based, meaning that '0 == one non-admin queue',
     '1 == two non-admin queues', and so forth.

Because our `num_queues` means the number of queues _plus_ the admin
queue, then the right calculation for the number returned from the admin
command is `num_queues - 2`, combining the two requirements mentioned.

The issue was discovered by reducing num_queues from 64 to 8 and running
a Linux VM with an SMP parameter larger than that (e.g. 22). It tries to
utilize all queues, and therefore fails with an invalid queue number
when trying to queue I/Os on the last queue.

Signed-off-by: Dan Aloni <dan@kernelim.com>
CC: Alex Friedman <alex@e8storage.com>
CC: Keith Busch <keith.busch@intel.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
---
 hw/block/nvme.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 7428db9f0c91..08ddf3a39e2f 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -573,7 +573,7 @@ static uint16_t nvme_get_feature(NvmeCtrl *n, NvmeCmd *cmd, NvmeRequest *req)
         result = blk_enable_write_cache(n->conf.blk);
         break;
     case NVME_NUMBER_OF_QUEUES:
-        result = cpu_to_le32((n->num_queues - 1) | ((n->num_queues - 1) << 16));
+        result = cpu_to_le32((n->num_queues - 2) | ((n->num_queues - 2) << 16));
         break;
     default:
         return NVME_INVALID_FIELD | NVME_DNR;
@@ -594,7 +594,7 @@ static uint16_t nvme_set_feature(NvmeCtrl *n, NvmeCmd *cmd, NvmeRequest *req)
         break;
     case NVME_NUMBER_OF_QUEUES:
         req->cqe.result =
-            cpu_to_le32((n->num_queues - 1) | ((n->num_queues - 1) << 16));
+            cpu_to_le32((n->num_queues - 2) | ((n->num_queues - 2) << 16));
         break;
     default:
         return NVME_INVALID_FIELD | NVME_DNR;
-- 
2.9.4

Re: [Qemu-devel] [PATCH] nvme: Fix get/set number of queues feature, again

[Keith Busch]

On Sun, May 28, 2017 at 04:06:49PM +0300, Dan Aloni wrote:
> The number of queues that should be return by the admin command should:
> 
>   1) Only mention the number of non-admin queues.
>   2) It is zero-based, meaning that '0 == one non-admin queue',
>      '1 == two non-admin queues', and so forth.
> 
> Because our `num_queues` means the number of queues _plus_ the admin
> queue, then the right calculation for the number returned from the admin
> command is `num_queues - 2`, combining the two requirements mentioned.
> 
> The issue was discovered by reducing num_queues from 64 to 8 and running
> a Linux VM with an SMP parameter larger than that (e.g. 22). It tries to
> utilize all queues, and therefore fails with an invalid queue number
> when trying to queue I/Os on the last queue.
> 
> Signed-off-by: Dan Aloni <dan@kernelim.com>
> CC: Alex Friedman <alex@e8storage.com>
> CC: Keith Busch <keith.busch@intel.com>
> CC: Stefan Hajnoczi <stefanha@redhat.com>

Looks good, thanks for the fix.

Reviewed-by: Keith Busch <keith.busch@intel.com>

Re: [Qemu-devel] [PATCH] nvme: Fix get/set number of queues feature, again

[Christoph Hellwig]

This didn't seem to make it into mainline, does it need a ping?

Re: [Qemu-devel] [PATCH] nvme: Fix get/set number of queues feature, again

[Stefan Hajnoczi]

On Sun, May 28, 2017 at 04:06:49PM +0300, Dan Aloni wrote:
> The number of queues that should be return by the admin command should:
> 
>   1) Only mention the number of non-admin queues.
>   2) It is zero-based, meaning that '0 == one non-admin queue',
>      '1 == two non-admin queues', and so forth.
> 
> Because our `num_queues` means the number of queues _plus_ the admin
> queue, then the right calculation for the number returned from the admin
> command is `num_queues - 2`, combining the two requirements mentioned.
> 
> The issue was discovered by reducing num_queues from 64 to 8 and running
> a Linux VM with an SMP parameter larger than that (e.g. 22). It tries to
> utilize all queues, and therefore fails with an invalid queue number
> when trying to queue I/Os on the last queue.
> 
> Signed-off-by: Dan Aloni <dan@kernelim.com>
> CC: Alex Friedman <alex@e8storage.com>
> CC: Keith Busch <keith.busch@intel.com>
> CC: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  hw/block/nvme.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)

Sorry for the delay.  Please send future nvme.c patches through Kevin
Wolf:

$ scripts/get_maintainer.pl -f hw/block/nvme.c
Keith Busch <keith.busch@intel.com> (supporter:nvme)
Kevin Wolf <kwolf@redhat.com> (supporter:Block layer core)
Max Reitz <mreitz@redhat.com> (supporter:Block layer core)
qemu-block@nongnu.org (open list:nvme)
qemu-devel@nongnu.org (open list:All patches CC here)

This patch will be in QEMU 2.11.

Thanks, applied to my block-next tree:
https://github.com/stefanha/qemu/commits/block-next

Stefan