Re: [PATCH 1/1] nvme: always enable multipath

[Nilay Shroff <nilay@linux.ibm.com>]

[-- Attachment #1: Type: text/plain, Size: 1880 bytes --]



On 11/22/24 19:40, John Meneghini wrote:
> On 11/22/24 01:26, Nilay Shroff wrote:
>>  
>> On 11/22/24 03:33, Bryan Gurney wrote:
>>
>> I applied the above changes to my kernel tree and ran the below blktests:
>>
>> # ./check nvme/033 nvme/034 nvme/035 nvme/036 nvme/037 nvme/039
>> nvme/033 => nvme0n1 (tr=loop) (create and connect to an NVMeOF target with a passthru controller) [not run]
>>      /dev/nvme0n1 is a NVMe multipath device
>> nvme/034 => nvme0n1 (tr=loop) (run data verification fio job on an NVMeOF passthru controller) [not run]
>>      /dev/nvme0n1 is a NVMe multipath device
>> nvme/035 => nvme0n1 (tr=loop) (run mkfs and data verification fio job on an NVMeOF passthru controller) [not run]
>>      /dev/nvme0n1 is a NVMe multipath device
>> nvme/036 => nvme0n1 (tr=loop) (test NVMe reset command on an NVMeOF target with a passthru controller) [not run]
>>      /dev/nvme0n1 is a NVMe multipath device
>> nvme/037 => nvme0n1 (tr=loop) (test deletion of NVMeOF passthru controllers immediately after setup) [not run]
>>      /dev/nvme0n1 is a NVMe multipath device
>> nvme/039 => nvme0n1 (test error logging)                     [not run]
>>      /dev/nvme0n1 is a NVMe multipath device
>>
>> As we can see here, the above tests were skipped because the test detects
>> that the device (/dev/nvme0n1) is a multipath device. However, in fact,
>> the test device is NOT a multipath. So I think we need to update the above
>> tests. We may submit another patch to blktest and update above tests once
>> your changes are merged upstream.
> 
> Nilay, what kind of device is /dev/nvme0n1 ?
> 
It's an NVMe namespace attached to a controller. 
> Can you show us the output of
> 
> nvme id-ctrl /dev/nvme0 -H
> nvme id-ns /dev/nvme0n1 -H
> 
Sure, attached the output of above commands for your reference.

Thanks,
--Nilay

[-- Attachment #2: nvme-id.txt --]
[-- Type: text/plain, Size: 13087 bytes --]

# nvme list -v 
Subsystem        Subsystem-NQN                                                                                    Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------
nvme-subsys0     nqn.1994-11.com.samsung:nvme:PM1735:2.5-inch:S6EUNA0R500358                                      nvme0

Device           Cntlid SN                   MN                                       FR       TxPort Address        Slot   Subsystem    Namespaces      
---------------- ------ -------------------- ---------------------------------------- -------- ------ -------------- ------ ------------ ----------------
nvme0    65     S6EUNA0R500358       1.6TB NVMe Gen4 U.2 SSD                  REV.SN49 pcie   0018:01:00.0   U78D4.ND0.WZS0021-P1-C8 nvme-subsys0 nvme0n1

Device            Generic           NSID       Usage                      Format           Controllers     
----------------- ----------------- ---------- -------------------------- ---------------- ----------------
/dev/nvme0n1 /dev/ng0n1   0x1          5.75  GB /   5.75  GB      4 KiB +  8 B   nvme0



# nvme id-ctrl /dev/nvme0 -H 
NVME Identify Controller:
vid       : 0x144d
ssvid     : 0x1014
sn        : S6EUNA0R500358      
mn        : 1.6TB NVMe Gen4 U.2 SSD                 
fr        : REV.SN49
rab       : 8
ieee      : 002538
cmic      : 0x3
  [3:3] : 0	ANA not supported
  [2:2] : 0	PCI
  [1:1] : 0x1	Multi Controller
  [0:0] : 0x1	Multi Port

mdts      : 0
cntlid    : 0x41
ver       : 0x10300
rtd3r     : 0xe4e1c0
rtd3e     : 0x989680
oaes      : 0x300
  [31:31] : 0	Discovery Log Change Notice Not Supported
  [27:27] : 0	Zone Descriptor Changed Notices Not Supported
  [16:16] : 0	Temperature Threshold Hysteresis Recovery Not Supported
  [15:15] : 0	Normal NSS Shutdown Event Not Supported
  [14:14] : 0	Endurance Group Event Aggregate Log Page Change Notice Not Supported
  [13:13] : 0	LBA Status Information Notices Not Supported
  [12:12] : 0	Predictable Latency Event Aggregate Log Change Notices Not Supported
  [11:11] : 0	Asymmetric Namespace Access Change Notices Not Supported
  [9:9] : 0x1	Firmware Activation Notices Supported
  [8:8] : 0x1	Namespace Attribute Changed Event Supported

ctratt    : 0
  [19:19] : 0	Flexible Data Placement Not Supported
  [17:17] : 0	HMB Restrict Non-Operational Power State Access Not Supported
  [16:16] : 0	MDTS and Size Limits Exclude Metadata Not Supported
  [15:15] : 0	Extended LBA Formats Not Supported
  [14:14] : 0	Delete NVM Set Not Supported
  [13:13] : 0	Delete Endurance Group Not Supported
  [12:12] : 0	Variable Capacity Management Not Supported
  [11:11] : 0	Fixed Capacity Management Not Supported
  [10:10] : 0	Multi Domain Subsystem Not Supported
  [9:9] : 0	UUID List Not Supported
  [8:8] : 0	SQ Associations Not Supported
  [7:7] : 0	Namespace Granularity Not Supported
  [6:6] : 0	Traffic Based Keep Alive Not Supported
  [5:5] : 0	Predictable Latency Mode Not Supported
  [4:4] : 0	Endurance Groups Not Supported
  [3:3] : 0	Read Recovery Levels Not Supported
  [2:2] : 0	NVM Sets Not Supported
  [1:1] : 0	Non-Operational Power State Permissive Not Supported
  [0:0] : 0	128-bit Host Identifier Not Supported

rrls      : 0
cntrltype : 0
  [7:2] : 0	Reserved
  [1:0] : 0	Controller type not reported
fguid     : 00000000-0000-0000-0000-000000000000
crdt1     : 0
crdt2     : 0
crdt3     : 0
nvmsr     : 1
  [1:1] : 0	NVM subsystem Not part of an Enclosure
  [0:0] : 0x1	NVM subsystem part of a Storage Device

vwci      : 255
  [7:7] : 0x1	VPD Write Cycles Remaining field is valid.
  [6:0] : 0xfe	VPD Write Cycles Remaining

mec       : 0
  [1:1] : 0	NVM subsystem Not contains a Management Endpoint on a PCIe port
  [0:0] : 0	NVM subsystem Not contains a Management Endpoint on an SMBus/I2C port

oacs      : 0x5f
  [10:10] : 0	Lockdown Command and Feature Not Supported
  [9:9] : 0	Get LBA Status Capability Not Supported
  [8:8] : 0	Doorbell Buffer Config Not Supported
  [7:7] : 0	Virtualization Management Not Supported
  [6:6] : 0x1	NVMe-MI Send and Receive Supported
  [5:5] : 0	Directives Not Supported
  [4:4] : 0x1	Device Self-test Supported
  [3:3] : 0x1	NS Management and Attachment Supported
  [2:2] : 0x1	FW Commit and Download Supported
  [1:1] : 0x1	Format NVM Supported
  [0:0] : 0x1	Security Send and Receive Supported

acl       : 127
aerl      : 15
frmw      : 0x16
  [5:5] : 0	Multiple FW or Boot Update Detection Not Supported
  [4:4] : 0x1	Firmware Activate Without Reset Supported
  [3:1] : 0x3	Number of Firmware Slots
  [0:0] : 0	Firmware Slot 1 Read/Write

lpa       : 0x1e
  [6:6] : 0	Telemetry Log Data Area 4 Not Supported
  [5:5] : 0	LID 0x0, Scope of each command in LID 0x5, 0x12, 0x13 Not Supported
  [4:4] : 0x1	Persistent Event log Supported
  [3:3] : 0x1	Telemetry host/controller initiated log page Supported
  [2:2] : 0x1	Extended data for Get Log Page Supported
  [1:1] : 0x1	Command Effects Log Page Supported
  [0:0] : 0	SMART/Health Log Page per NS Not Supported

elpe      : 255
  [7:0] : 255 (0's based)	Error Log Page Entries (ELPE)

npss      : 2
  [7:0] : 2 (0's based)	Number of Power States Support (NPSS)

avscc     : 0x1
  [0:0] : 0x1	Admin Vendor Specific Commands uses NVMe Format

apsta     : 0
  [0:0] : 0	Autonomous Power State Transitions Not Supported

wctemp    : 345
 [15:0] : 72 °C (345 K)	Warning Composite Temperature Threshold (WCTEMP)

cctemp    : 358
 [15:0] : 85 °C (358 K)	Critical Composite Temperature Threshold (CCTEMP)

mtfa      : 130
hmpre     : 0
hmmin     : 0
tnvmcap   : 1600321314816
[127:0] : 1600321314816
	Total NVM Capacity (TNVMCAP)

unvmcap   : 1594569523200
[127:0] : 1594569523200
	Unallocated NVM Capacity (UNVMCAP)

rpmbs     : 0
 [31:24]: 0	Access Size
 [23:16]: 0	Total Size
  [5:3] : 0	Authentication Method
  [2:0] : 0	Number of RPMB Units

edstt     : 2
dsto      : 1
fwug      : 255
kas       : 10
hctma     : 0
  [0:0] : 0	Host Controlled Thermal Management Not Supported

mntmt     : 0
 [15:0] : -273 °C (0 K)	Minimum Thermal Management Temperature (MNTMT)

mxtmt     : 0
 [15:0] : -273 °C (0 K)	Maximum Thermal Management Temperature (MXTMT)

sanicap   : 0x3
  [31:30] : 0	Additional media modification after sanitize operation completes successfully is not defined
  [29:29] : 0	No-Deallocate After Sanitize bit in Sanitize command Supported
  [3:3] : 0	Media Verification and Post-Verification Deallocation state Not Supported
  [2:2] : 0	Overwrite Sanitize Operation Not Supported
  [1:1] : 0x1	Block Erase Sanitize Operation Supported
  [0:0] : 0x1	Crypto Erase Sanitize Operation Supported

hmminds   : 0
hmmaxd    : 0
nsetidmax : 0
endgidmax : 0
anatt     : 0
anacap    : 0
  [7:7] : 0	Non-zero group ID Not Supported
  [6:6] : 0	Group ID does change
  [4:4] : 0	ANA Change state Not Supported
  [3:3] : 0	ANA Persistent Loss state Not Supported
  [2:2] : 0	ANA Inaccessible state Not Supported
  [1:1] : 0	ANA Non-optimized state Not Supported
  [0:0] : 0	ANA Optimized state Not Supported

anagrpmax : 0
nanagrpid : 0
pels      : 80
domainid  : 0
megcap    : 0
tmpthha   : 0
  [2:0] : 0	Temperature Threshold Maximum Hysteresis

sqes      : 0x66
  [7:4] : 0x6	Max SQ Entry Size (64)
  [3:0] : 0x6	Min SQ Entry Size (64)

cqes      : 0x44
  [7:4] : 0x4	Max CQ Entry Size (16)
  [3:0] : 0x4	Min CQ Entry Size (16)

maxcmd    : 0
nn        : 64
oncs      : 0x7f
  [10:10] : 0	All Fast Copy Not Supported
  [9:9] : 0	Copy Single Atomicity Not Supported
  [8:8] : 0	Copy Not Supported
  [7:7] : 0	Verify Not Supported
  [6:6] : 0x1	Timestamp Supported
  [5:5] : 0x1	Reservations Supported
  [4:4] : 0x1	Save and Select Supported
  [3:3] : 0x1	Write Zeroes Supported
  [2:2] : 0x1	Data Set Management Supported
  [1:1] : 0x1	Write Uncorrectable Supported
  [0:0] : 0x1	Compare Supported

fuses     : 0
  [0:0] : 0	Fused Compare and Write Not Supported

fna       : 0x4
  [3:3] : 0	Format NVM Broadcast NSID (FFFFFFFFh) Supported
  [2:2] : 0x1	Crypto Erase Supported as part of Secure Erase
  [1:1] : 0	Crypto Erase Applies to Single Namespace(s)
  [0:0] : 0	Format Applies to Single Namespace(s)

vwc       : 0
  [2:1] : 0	Support for the NSID field set to FFFFFFFFh is not indicated
  [0:0] : 0	Volatile Write Cache Not Present

awun      : 65535
awupf     : 0
icsvscc   : 1
  [0:0] : 0x1	NVM Vendor Specific Commands uses NVMe Format

nwpc      : 0
  [2:2] : 0	Permanent Write Protect Not Supported
  [1:1] : 0	Write Protect Until Power Supply Not Supported
  [0:0] : 0	No Write Protect and Write Protect Namespace Not Supported

acwu      : 0
ocfs      : 0
  [3:3] : 0	Controller Copy Format 3h Not Supported
  [2:2] : 0	Controller Copy Format 2h Not Supported
  [1:1] : 0	Controller Copy Format 1h Not Supported
  [0:0] : 0	Controller Copy Format 0h Not Supported

sgls      : 0xf0002
 [21:21]: 0	Transport SGL Data Block Descriptor Not Supported
 [20:20]: 0	Address Offsets Not Supported
 [19:19]: 0x1	Metadata Pointer Containing SGL Descriptor is Supported
 [18:18]: 0x1	SGL Length Larger than Buffer Supported
 [17:17]: 0x1	Byte-Aligned Contig. MD Buffer Supported
 [16:16]: 0x1	SGL Bit-Bucket Supported
 [15:8] : 0	SGL Descriptor Threshold
  [2:2] : 0	Keyed SGL Data Block descriptor Not Supported
  [1:0] : 0x2	Scatter-Gather Lists Supported. Dword alignment required.

mnan      : 0
maxdna    : 0
maxcna    : 0
oaqd      : 0
subnqn    : nqn.1994-11.com.samsung:nvme:PM1735:2.5-inch:S6EUNA0R500358      
ioccsz    : 0
iorcsz    : 0
icdoff    : 0
fcatt     : 0
  [0:0] : 0	Dynamic Controller Model

msdbd     : 0
ofcs      : 0
  [0:0] : 0	Disconnect command Not Supported

ps      0 : mp:25.00W operational enlat:180 exlat:180 rrt:0 rrl:0
            rwt:0 rwl:0 idle_power:7.00W active_power:15.00W
            active_power_workload:80K 128KiB SW
ps      1 : mp:18.00W operational enlat:180 exlat:180 rrt:1 rrl:1
            rwt:1 rwl:1 idle_power:7.00W active_power:15.00W
            active_power_workload:80K 128KiB SW
ps      2 : mp:12.00W operational enlat:180 exlat:180 rrt:2 rrl:2
            rwt:2 rwl:2 idle_power:7.00W active_power:12.00W
            active_power_workload:80K 128KiB SW



# nvme id-ns /dev/nvme0n1 -H 
NVME Identify Namespace 1:
nsze    : 0x156d56
ncap    : 0x156d56
nuse    : 0x156d56
nsfeat  : 0
  [5:4] : 0	NPWG, NPWA, NPDG, NPDGL, NPDA, and NOWS are Not Supported
  [3:3] : 0	NGUID and EUI64 fields if non-zero, Reused
  [2:2] : 0	Deallocated or Unwritten Logical Block error Not Supported
  [1:1] : 0	Namespace uses AWUN, AWUPF, and ACWU
  [0:0] : 0	Thin Provisioning Not Supported

nlbaf   : 4
flbas   : 0x1
  [6:5] : 0	Most significant 2 bits of Current LBA Format Selected
  [4:4] : 0	Metadata Transferred in Separate Contiguous Buffer
  [3:0] : 0x1	Least significant 4 bits of Current LBA Format Selected

mc      : 0x3
  [1:1] : 0x1	Metadata Pointer Supported
  [0:0] : 0x1	Metadata as Part of Extended Data LBA Supported

dpc     : 0x1f
  [4:4] : 0x1	Protection Information Transferred as Last Bytes of Metadata Supported
  [3:3] : 0x1	Protection Information Transferred as First Bytes of Metadata Supported
  [2:2] : 0x1	Protection Information Type 3 Supported
  [1:1] : 0x1	Protection Information Type 2 Supported
  [0:0] : 0x1	Protection Information Type 1 Supported

dps     : 0x1
  [3:3] : 0	Protection Information is Transferred as Last Bytes of Metadata
  [2:0] : 0x1	Protection Information Type 1 Enabled

nmic    : 0
  [0:0] : 0	Namespace Multipath Not Capable

rescap  : 0xff
  [7:7] : 0x1	Ignore Existing Key - Used as defined in revision 1.3 or later
  [6:6] : 0x1	Exclusive Access - All Registrants Supported
  [5:5] : 0x1	Write Exclusive - All Registrants Supported
  [4:4] : 0x1	Exclusive Access - Registrants Only Supported
  [3:3] : 0x1	Write Exclusive - Registrants Only Supported
  [2:2] : 0x1	Exclusive Access Supported
  [1:1] : 0x1	Write Exclusive Supported
  [0:0] : 0x1	Persist Through Power Loss Supported

fpi     : 0x80
  [7:7] : 0x1	Format Progress Indicator Supported
  [6:0] : 0	Format Progress Indicator (Remaining 0%)

dlfeat  : 17
  [4:4] : 0x1	Guard Field of Deallocated Logical Blocks is set to CRC of The Value Read
  [3:3] : 0	Deallocate Bit in the Write Zeroes Command is Not Supported
  [2:0] : 0x1	Bytes Read From a Deallocated Logical Block and its Metadata are 0x00

nawun   : 0
nawupf  : 0
nacwu   : 0
nabsn   : 0
nabo    : 0
nabspf  : 0
noiob   : 0
nvmcap  : 5751791616
mssrl   : 0
mcl     : 0
msrc    : 0
nulbaf  : 0
anagrpid: 0
nsattr	: 0
nvmsetid: 0
endgid  : 0
nguid   : 36455530525003580025384100000001
eui64   : 0000000000000000
LBA Format  0 : Metadata Size: 0   bytes - Data Size: 4096 bytes - Relative Performance: 0 Best 
LBA Format  1 : Metadata Size: 8   bytes - Data Size: 4096 bytes - Relative Performance: 0x2 Good (in use)
LBA Format  2 : Metadata Size: 0   bytes - Data Size: 512 bytes - Relative Performance: 0x1 Better 
LBA Format  3 : Metadata Size: 8   bytes - Data Size: 512 bytes - Relative Performance: 0x3 Degraded 
LBA Format  4 : Metadata Size: 64  bytes - Data Size: 4096 bytes - Relative Performance: 0x3 Degraded