Re: [PATCH v7 00/18] NVMe PCI endpoint target driver

Manivannan Sadhasivam
2024-12-20T12:22:37Z

On Fri, Dec 20, 2024 at 06:50:50PM +0900, Damien Le Moal wrote:
> This patch series implements an NVMe target driver for the PCI transport
> using the PCI endpoint framework.
>
> The first 5 patches of this series move and cleanup some nvme code that
> will be reused in following patches.
>
> Patch 6 introduces the PCI transport type to allow setting up ports for
> the new PCI target controller driver. Patch 7 to 10 are improvements of
> the target core code to allow creating the PCI controller and processing
> its nvme commands without the need to rely on fabrics commands like the
> connect command to create the admin and I/O queues.
>
> Patch 11 relaxes the SGL check in nvmet_req_init() to allow for PCI
> admin commands (which must use PRPs).
>
> Patches 12 to 16 improve the set/get feature support of the target code
> to get closer to achieving NVMe specification compliance. These patches
> though do not implement support for some mandatory features.
>
> Patch 17 is the main patch which introduces the NVMe PCI endpoint target
> driver. This patch commit message provides and overview of the driver
> design and operation.
>
> Finally, patch 18 documents the NVMe PCI endpoint target driver and
> provides a user guide explaning how to setup an NVMe PCI endpoint
> device.
>
> The patches are base on Linus 6.13-rc3 tree.
>
> This driver has been extensively tested using a Radxa Rock5B board
> (RK3588 Arm SoC). Some tests have also been done using a Pine Rockpro64
> board. However, this board does not support DMA channels for the PCI
> endpoint controller, leading to very poor performance.
>
> Using the Radxa Rock5b board and setting up a 4 queue-pairs controller
> with a null-blk block device loop target, performance was measured using
> fio as follows:
>
> +----------------------------------+------------------------+
> | Workload | IOPS (BW) |
> +----------------------------------+------------------------+
> | Rand read, 4KB, QD=1, 1 job | 14.3k IOPS |
> | Rand read, 4KB, QD=32, 1 job | 80.8k IOPS |
> | Rand read, 4KB, QD=32, 4 jobs | 131k IOPS |
> | Rand read, 128KB, QD=32, 1 job | 16.7k IOPS (2.18 GB/s) |
> | Rand read, 128KB, QD=32, 4 jobs | 17.4k IOPS (2.27 GB/s) |
> | Rand read, 512KB, QD=32, 1 job | 5380 IOPS (2.82 GB/s) |
> | Rand read, 512KB, QD=32, 4 jobs | 5206 IOPS (2.27 GB/s) |
> | Rand write, 128KB, QD=32, 1 job | 9617 IOPS (1.26 GB/s) |
> | Rand write, 128KB, QD=32, 4 jobs | 8405 IOPS (1.10 GB/s) |
> +----------------------------------+------------------------+
>
> These results use the default MDTS of the NVMe enpoint driver of 512 KB.
>
> This driver is not intended for production use but rather to be a
> playground for learning NVMe and exploring/testing new NVMe features
> while providing reasonably good performance.
>

Series looks good from PCI endpoint perspective and I've given my R-b tags. I
hope this gets merged through the nvme tree.

Thanks a lot for the work, Damien and Rick!

- Mani

--
மணிவண்ணன் சதாசிவம்