[PATCH] NVMe: increase depth of admin queue

[PATCH] NVMe: increase depth of admin queue

[Jens Axboe]

Hi,

Usually the admin queue depth of 64 is plenty, but for some use cases we
really need it larger. Examples are use cases like MAT, where you have
to touch all of NAND for init/format like purposes. In those cases, we
see a good 2x increase with an increased queue depth.

Signed-off-by: Jens Axboe <axboe at fb.com>

diff --git a/drivers/block/nvme-core.c b/drivers/block/nvme-core.c
index ceb32dd52a6c..b67b520e489d 100644
--- a/drivers/block/nvme-core.c
+++ b/drivers/block/nvme-core.c
@@ -44,7 +44,7 @@
 
 #define NVME_MINORS		(1U << MINORBITS)
 #define NVME_Q_DEPTH		1024
-#define NVME_AQ_DEPTH		64
+#define NVME_AQ_DEPTH		256
 #define SQ_SIZE(depth)		(depth * sizeof(struct nvme_command))
 #define CQ_SIZE(depth)		(depth * sizeof(struct nvme_completion))
 #define ADMIN_TIMEOUT		(admin_timeout * HZ)

-- 
Jens Axboe

[PATCH] NVMe: increase depth of admin queue

[J Freyensee]

On Fri, 2015-03-06@12:56 -0700, Jens Axboe wrote:
> Hi,
> 
> Usually the admin queue depth of 64 is plenty, but for some use cases we
> really need it larger. Examples are use cases like MAT, where you have
> to touch all of NAND for init/format like purposes. In those cases, we
> see a good 2x increase with an increased queue depth.
> 
> Signed-off-by: Jens Axboe <axboe at fb.com>
> 
> diff --git a/drivers/block/nvme-core.c b/drivers/block/nvme-core.c
> index ceb32dd52a6c..b67b520e489d 100644
> --- a/drivers/block/nvme-core.c
> +++ b/drivers/block/nvme-core.c
> @@ -44,7 +44,7 @@
>  
>  #define NVME_MINORS		(1U << MINORBITS)
>  #define NVME_Q_DEPTH		1024
> -#define NVME_AQ_DEPTH		64
> +#define NVME_AQ_DEPTH		256

Would this be better served to have it be a module parameter so someone
can change it to/from 64 to whatever depth they think they need for
their targeted system and workload?


>  #define SQ_SIZE(depth)		(depth * sizeof(struct nvme_command))
>  #define CQ_SIZE(depth)		(depth * sizeof(struct nvme_completion))
>  #define ADMIN_TIMEOUT		(admin_timeout * HZ)
> 

[PATCH] NVMe: increase depth of admin queue

[Keith Busch]

On Fri, 6 Mar 2015, Jens Axboe wrote:
> Usually the admin queue depth of 64 is plenty, but for some use cases we
> really need it larger. Examples are use cases like MAT, where you have
> to touch all of NAND for init/format like purposes. In those cases, we
> see a good 2x increase with an increased queue depth.
>
> Signed-off-by: Jens Axboe <axboe at fb.com>

Sounds good to me. If we encounter a case where changing this again is
desirable, we might want to take Jay's suggestion to make it a module
param.

Acked-by: Keith Busch <keith.busch at intel.com>