[PATCH] nvme-pci: derive and better document max segments limits

[PATCH] nvme-pci: derive and better document max segments limits

[Christoph Hellwig]

Redefine the max segments and max integrity limits based on the limiting
factors without changing the final values and add comments explaining
them.

Signed-off-by: Christoph Hellwig <hch@lst.de>
---
 drivers/nvme/host/pci.c | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 0efbc9329291..0b6e54f7ba05 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -35,8 +35,6 @@
 #define SQ_SIZE(q)	((q)->q_depth << (q)->sqes)
 #define CQ_SIZE(q)	((q)->q_depth * sizeof(struct nvme_completion))
 
-#define SGES_PER_PAGE	(NVME_CTRL_PAGE_SIZE / sizeof(struct nvme_sgl_desc))
-
 #define NVME_SMALL_POOL_SIZE	256
 
 /*
@@ -44,10 +42,24 @@
  * require an sg allocation that needs more than a page of data.
  */
 #define NVME_MAX_KB_SZ	8192
-#define NVME_MAX_SEGS	128
-#define NVME_MAX_META_SEGS 15
 #define NVME_MAX_NR_DESCRIPTORS	5
 
+/*
+ * For data SGLs we support a single descriptors worth of SGL entries, but for
+ * now we also limit it to avoid an allocation larger than PAGE_SIZE for the
+ * scatterlist.
+ */
+#define NVME_MAX_SEGS \
+	min(NVME_CTRL_PAGE_SIZE / sizeof(struct nvme_sgl_desc), \
+	    (PAGE_SIZE / sizeof(struct scatterlist)))
+
+/*
+ * For metadata SGLs, only the small descriptor is supported, and the first
+ * entry is the segment descriptor, which for the data pointer sits in the SQE.
+ */
+#define NVME_MAX_META_SEGS \
+	((NVME_SMALL_POOL_SIZE / sizeof(struct nvme_sgl_desc)) - 1)
+
 static int use_threaded_interrupts;
 module_param(use_threaded_interrupts, int, 0444);
 
@@ -3829,8 +3841,6 @@ static int __init nvme_init(void)
 	BUILD_BUG_ON(sizeof(struct nvme_create_sq) != 64);
 	BUILD_BUG_ON(sizeof(struct nvme_delete_queue) != 64);
 	BUILD_BUG_ON(IRQ_AFFINITY_MAX_SETS < 2);
-	BUILD_BUG_ON(NVME_MAX_SEGS > SGES_PER_PAGE);
-	BUILD_BUG_ON(sizeof(struct scatterlist) * NVME_MAX_SEGS > PAGE_SIZE);
 	BUILD_BUG_ON(nvme_pci_npages_prp() > NVME_MAX_NR_DESCRIPTORS);
 
 	return pci_register_driver(&nvme_driver);
-- 
2.47.2

Re: [PATCH] nvme-pci: derive and better document max segments limits

[Keith Busch]

On Fri, May 16, 2025 at 09:21:52AM +0200, Christoph Hellwig wrote:
> Redefine the max segments and max integrity limits based on the limiting
> factors without changing the final values and add comments explaining
> them.

...
 
>  #define NVME_MAX_KB_SZ	8192
> -#define NVME_MAX_SEGS	128
> -#define NVME_MAX_META_SEGS 15
>  #define NVME_MAX_NR_DESCRIPTORS	5
>  
> +/*
> + * For data SGLs we support a single descriptors worth of SGL entries, but for
> + * now we also limit it to avoid an allocation larger than PAGE_SIZE for the
> + * scatterlist.
> + */
> +#define NVME_MAX_SEGS \
> +	min(NVME_CTRL_PAGE_SIZE / sizeof(struct nvme_sgl_desc), \
> +	    (PAGE_SIZE / sizeof(struct scatterlist)))

If your page size is >=8k, this should change the value of NVME_MAX_SEGS
to 256. I don't think that's a bad thing, but the commit message doesn't
reflect that.

The diff looks good.

Reviewed-by: Keith Busch <kbusch@kernel.org>

Re: [PATCH] nvme-pci: derive and better document max segments limits

[Christoph Hellwig]

On Fri, May 16, 2025 at 09:17:26AM -0600, Keith Busch wrote:
> If your page size is >=8k, this should change the value of NVME_MAX_SEGS
> to 256. I don't think that's a bad thing, but the commit message doesn't
> reflect that.

Indeed.  I've update the commit log and pushed the changes to nvme-6.16.

Re: [PATCH] nvme-pci: derive and better document max segments limits

[Chaitanya Kulkarni]

On 5/16/25 00:21, Christoph Hellwig wrote:
> Redefine the max segments and max integrity limits based on the limiting
> factors without changing the final values and add comments explaining
> them.
>
> Signed-off-by: Christoph Hellwig<hch@lst.de>


Looks good.

Reviewed-by: Chaitanya Kulkarni <kch@nvidia.com>

-ck