Re: [External] : Re: [bug-report] 5-9% FIO randomwrite ext4 perf regression on 6.12.y kernel

[Paul Webb <paul.x.webb@oracle.com>]

On 22/11/2024 12:13, Christoph Hellwig wrote:
> On Thu, Nov 21, 2024 at 09:07:32PM +0000, Paul Webb wrote:
>> Christoph:
>> To check for weird lazy init code using write zeroes
>>
>> Values in the 5.15 kernel baseline prior to the commit:
>> $ cat /sys/block/nvme*n1/queue/write_zeroes_max_bytes
>> 0
>> 0
>> 0
>> 0
>>
>> Values in the 6.11 kernel that contains the commit:
>> $ cat /sys/block/nvme*n1/queue/write_zeroes_max_bytes
>> 2199023255040
>> 2199023255040
>> 2199023255040
>> 2199023255040
> Thanks!  So 6.11 actually enables write zeroes for your controller.
>
>> Another interesting datapoint is that while performing some runs I am
>> seeing the following output on the console in the 6.11/6.12 kernels that
>> contain the commit:
>>
>> [  473.398188] operation not supported error, dev nvme2n1, sector 13952 op 0x9:(WRITE_ZEROES) flags 0x800 phys_seg 0 prio class 0
> .. which it doesn't handle well.
>
>> [  473.534550] nvme0n1: Dataset Management(0x9) @ LBA 14000, 256 blocks, Invalid Command Opcode (sct 0x0 / sc 0x1) DNR
> .. and interesting this is for a Deallocate, which should only happen
> with the quirk for certain Intel controllers from the very first days of
> nvme.
>
> What controller do you have?  Can you post the output of lspci and
> "nvme list"?

Hi Christoph,

The nvme related output from lspci is as follows:
$ lspci | grep -i nvme
19:00.0 Non-Volatile memory controller: Intel Corporation NVMe 
Datacenter SSD [3DNAND, Beta Rock Controller]
20:00.0 Non-Volatile memory controller: Intel Corporation NVMe 
Datacenter SSD [3DNAND, Beta Rock Controller]
94:00.0 Non-Volatile memory controller: Intel Corporation NVMe 
Datacenter SSD [3DNAND, Beta Rock Controller]
9b:00.0 Non-Volatile memory controller: Intel Corporation NVMe 
Datacenter SSD [3DNAND, Beta Rock Controller]


$ sudo nvme list
Node                  Generic               SN 
Model                                    Namespace 
Usage                      Format           FW Rev
--------------------- --------------------- -------------------- 
---------------------------------------- ---------- 
-------------------------- ---------------- --------
/dev/nvme0n1          /dev/ng0n1            PHLN942100EQ6P4CGN 
7361456_ICRPC2DD2ORA6.4T                 0x1          6.40  TB / 6.40  
TB    512   B +  0 B   VDV1RL06
/dev/nvme1n1          /dev/ng1n1            PHLN942100PE6P4CGN 
7361456_ICRPC2DD2ORA6.4T                 0x1          6.40  TB / 6.40  
TB    512   B +  0 B   VDV1RL06
/dev/nvme2n1          /dev/ng2n1            PHLN9415002B6P4CGN 
7361456_ICRPC2DD2ORA6.4T                 0x1          6.40  TB / 6.40  
TB    512   B +  0 B   VDV1RL06
/dev/nvme3n1          /dev/ng3n1            PHLN942100DQ6P4CGN 
7361456_ICRPC2DD2ORA6.4T                 0x1          6.40  TB / 6.40  
TB    512   B +  0 B   VDV1RL06