[PATCH v1 0/1] nvme-pci: adaptive interrupt coalescing

[Jun Zeng <jun1.zeng@intel.com>]

Background
==========

Nowadays, with the high performance PCIe 5.0 NVMe drives been used
more increasingly and widely, it's important to maintain and fully
realize NVMe drive's performance at the system level in
scenarios where multiple NVMe drives are used simultaneously.

Problem
=======

With PCIe 5.0 NVMe drives, the interrupt count can reach very high
levels in burst when running IOPS workloads (e.g., 4k/8k randread)
with multiple queues across many NVMe drives. These high interrupts
generated instantaneously may put pressure on current CPU handling
mechanism.

Specifically, IOPS with small IO (e.g., 4k/8k randread) does not
meet expectations when running with multiple queues among many NVMe
drives (e.g., 6-8 NVMe drives per socket), even with each queue and
job assigned to separate CPU cores.

The IOPS for small IO can be significantly improved when enabling
interrupt coalescing in this case. However, in some cases (e.g., big
IO for bandwidth or latency-sensitive tests with small IO and small
queue depth), interrupt coalescing has side effects.

Solution
========

Monitor the IOPS and average queue depth of inflight IO periodically
on each NVMe drive and enable/disable interrupt coalescing
automatically without user intervention. This feature is enabled by
default and can be disabled/enabled through debugfs during runtime.

Test Results
============

1. IOPS improved from < 20M to 55M for 4k randread with fio job=8,
   qd=128 across 16 PCIe Gen5 NVMe drives on Intel Xeon GNR CPUs.
   - Latency not affected for 4k randread with job=1/2/4/8,
     qd=1/2/4/8/16/32
   - Bandwidth not affected for 32k/64k/128k read/write with
     job=1/2/4/8, qd=128

2. Tested with several mainstream PCIe Gen5 SSD vendors and Intel
   CPUs (including AMD) with improved 4K random read IOPS.

Notes
=====

1. Interrupt coalescing is a per-drive feature, not per-queue
   feature. Each queue has a separate interrupt, but coalescing is
   controlled globally per drive. We introduce thresholds for
   IO_DELTA, AVG_QUEUE_DEPTH, and TOTAL_PENDING_IO to judge
   workload status.

2. Tunable parameters exposed via debugfs at
   /sys/kernel/debug/nvme_pci/:
   - disable_adaptive_interrupt_coalescing: 0=enabled, 1=disabled
   - high_io_delta: IO count threshold per cycle (default: 45000)
   - avg_qd_threshold: average queue depth threshold (default: 32)
   - total_inflight_io_threshold: total inflight IO threshold
     (default: 256)

3. The default threshold values are derived from empirical testing
   on various PCIe Gen5 NVMe drives.

Jun Zeng (1):
  nvme-pci: adaptive interrupt coalescing

 drivers/nvme/host/pci.c | 540 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 539 insertions(+), 1 deletion(-)


base-commit: 5d6919055dec134de3c40167a490f33c74c12581
-- 
2.43.0