[Bug 219128] New: lpfc driver reports failure message

[Bug 219128] New: lpfc driver reports failure message

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

            Bug ID: 219128
           Summary: lpfc driver reports failure message
           Product: IO/Storage
           Version: 2.5
          Hardware: Intel
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P3
         Component: SCSI
          Assignee: linux-scsi@vger.kernel.org
          Reporter: lixc17@lenovo.com
        Regression: No

After installing Ubuntu 24.04 in a SAN disk via "Emulex LPe35000 32Gb 1-Port
Fibre Channel Adapter", the lpfc driver show many errors, like:

lpfc 0000:41:00.0: 53: [ 250.627110] 0:(0):0220 FDMI cmd failed FS_RJT Data:
x211
lpfc 0000:41:00.0: 0:(0):2754 PRLI failure DID:030E00 Status:x9/x32a00, data:
x4 x0 x80000000
lpfc 0000:41:00.0: 25: [ 247.537454] 0:(0):9024 FCP command xa3 failed: x0 SNS
x0 x0 Data: x8 xfdc x0 x0 x0
lpfc 0000:27:00.0: 0:(0):2753 PLOGI failure DID:040F00 Status:x3/x103
kernel: rport-13:0-5: blocked FC remote port time out: removing rport

Reproduce Steps:
  1. Install Ubunut 24.04 - can intall successfully
  2. Boot into OS
  3. Check /var/log/syslog, found lpfc relevance errors.
  4. Update to the upstream latest v6.10 kernel, the lpfc errors still there.

 Expected results:
 No lpfc driver related failures or errors.

 Actual results:
 Many lpfc driver failed messages was recorded.

Configuration:
LENOVO SR630V3 - Intel EGS
XCC: ESX323N
UEFI:ESE123F
UEFI Setting：Default

os:ubuntu-24.04-live-server-amd64-GA-0426
cpu: Intel(R) Xeon(R) Platinum 8460Y+*2
mem:M321R2GA3BB6-CQKEG*2
Emulex LPe35000 32Gb 1-Port Fibre Channel Adapter fw:  14.2.673.40
Intel Ethernet Server Adapter I350-T4 fw:  1.3450.0
SAN - Lenovo DM7000

Additional info:
  1. Although there're those failures, we can read/write date to SAN disks.
  2. Intel Birch Stream platform has this lpfc driver issues as well.

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 219128] lpfc driver reports failed messages

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

Xavier (lixc17@lenovo.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|lpfc driver reports failure |lpfc driver reports failed
                   |message                     |messages

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 219128] lpfc driver reports failed messages

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

--- Comment #1 from Xavier (lixc17@lenovo.com) ---
Created attachment 306673
  --> https://bugzilla.kernel.org/attachment.cgi?id=306673&action=edit
dmesg output log for upstream kernel 6.10

Errors still exist with 6.10 kernel, the attachment file is the kernel boot log
with parameter "lpfc.lpfc_log_verbose=4115 log_buf_len=32M"

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 219128] lpfc driver reports failed messages

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

--- Comment #2 from Xavier (lixc17@lenovo.com) ---
Link Ubuntu launchpad issue here at
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2073571

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 219128] lpfc driver reports failed messages

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

--- Comment #3 from justin.tee@broadcom.com ---
Hi Xavier,

Broadcom (Emulex) will have a look at this reported bug.

Please reassign to justin.tee@broadcom.com , and we will have a look at the
dmesg log.

Thanks,
Justin

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 219128] lpfc driver reports failed messages

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

--- Comment #4 from Xavier (lixc17@lenovo.com) ---
Hi, Justin
Thanks for your quick reply.
I found I don't have permission to change the bug assignee even I am the
reporter.

Regards,
--Xavier

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 219128] lpfc driver reports failed messages

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

--- Comment #5 from justin.tee@broadcom.com ---
(In reply to Xavier from comment #4)
> I found I don't have permission to change the bug assignee even I am the
> reporter.
Okay, no worries.  Please find below a reply based on dmesg log provided.

(In reply to Xavier from comment #1)
> Created attachment 306673 [details]
> dmesg output log for upstream kernel 6.10
> 
> Errors still exist with 6.10 kernel, the attachment file is the kernel boot
> log with parameter "lpfc.lpfc_log_verbose=4115 log_buf_len=32M"

1.) PRLI failure log messages.

[   12.754100] lpfc 0000:27:00.0: 0:(0):0108 No retry ELS command x18001420 to
remote NPORT x30700 Retried:1 Error:x9/b0000 IoTag x2f60 nflags x80020000
[   12.754102] lpfc 0000:27:00.0: 0:(0):2754 PRLI failure DID:030700
Status:x9/xb0000, data: x4 x0 x80020000
[   12.859313] lpfc 0000:27:00.0: 0:(0):2754 PRLI failure DID:020100
Status:x9/xb0000, data: x4 x0 x80020000
[   13.029796] lpfc 0000:27:00.0: 0:(0):2754 PRLI failure DID:051400
Status:x9/xb0000, data: x4 x0 x80000000
…
[   13.127530] lpfc 0000:27:00.0: 0:(0):2754 PRLI failure DID:030F00
Status:x9/xb0000, data: x4 x0 x80020000
[   13.127948] lpfc 0000:27:00.0: 0:(0):2754 PRLI failure DID:051000
Status:x9/xb0000, data: x4 x0 x80000000
[   13.129275] lpfc 0000:27:00.0: 0:(0):0108 No retry ELS command x18001420 to
remote NPORT x51601 Retried:1 Error:x9/b2c00 IoTag x2f48 nflags x80000000
[   13.129277] lpfc 0000:27:00.0: 0:(0):2754 PRLI failure DID:051601
Status:x9/xb2c00, data: x4 x0 x80000000
[   13.131082] lpfc 0000:27:00.0: 0:(0):2754 PRLI failure DID:051201
Status:x9/xb2c00, data: x4 x0 x80000000
…

These log messages indicate that several ports sent LS_RJT with reason code
0x0B (Command Unsupported) to our sent NVME-PRLI.  This is most likely due to a
target that does not support NVME.

2.) FDMI cmd failed messages.

[ 2236.658100] lpfc 0000:27:00.0: 0:(0):0220 FDMI cmd failed FS_RJT Data: x211

These log messages are benign and do not affect normal HBA operations.  It is
related to a proprietary management tool, which if interested, we can provide
further information in a non-public setting.

3.) FCP command failed messages.

[   12.501657] lpfc 0000:27:00.0: 0:(0):9024 FCP command xa0 failed: x0 SNS x0
x0 Data: x8 xff0 x0 x0 x0
[   12.526335] lpfc 0000:27:00.0: 0:(0):9024 FCP command x12 failed: x0 SNS x0
x0 Data: x8 xa x0 x0 x0

These are Report_LUNs and Inquiry SCSI commands during FCP discovery getting
logged with underruns.  This is part of normal HBA operations and can be
ignored because returned status in the SCSI commands are SAM_STAT_GOOD = 0x00.

4.) PLOGI failure messages.

[    9.707863] lpfc 0000:27:00.0: 0:(0):0116 Xmit ELS command x3 to remote
NPORT x30b00 I/O tag: x2fdd, port state:xd rpi xd fc_flag:x208114
…
[    9.707936] lpfc 0000:27:00.0: 0:(0):0112 ELS command x3 received from NPORT
x30b00 refcnt 3 Data: x20 x208154 x41000 x41000
[    9.707938] lpfc 0000:27:00.0: 0:(0):3178 PLOGI confirm: ndlp x30b00 x40000
x3: new_ndlp x0 x0 x0
[    9.707941] lpfc 0000:27:00.0: 0:(0):0211 DSM in event x0 on NPort x30b00 in
state 1 rpi xd Data: x40000 x30000
[    9.707943] lpfc 0000:27:00.0: 0:(0):0114 PLOGI chkparm OK Data: x30b00 x1
x40000 xd x20 x208154
[    9.707944] lpfc 0000:27:00.0: 0:(0):2819 Abort outstanding I/O on NPort
x30b00 Data: x40000 x1 xd
…
[   10.481216] lpfc 0000:27:00.0: 0:(0):0102 PLOGI completes to NPort x030b00
IoTag x2fdd Data: x3 x3 x103 x0 x1d
[   10.481218] lpfc 0000:27:00.0: 0:(0):0108 No retry ELS command x3 to remote
NPORT x30b00 Retried:0 Error:x3/103 IoTag x2fdd nflags x1080000
[   10.481219] lpfc 0000:27:00.0: 0:(0):2753 PLOGI failure DID:030B00
Status:x3/x103

This particular PLOGI was “failed” out because the lpfc driver originally sent
a PLOGI to port_id 0x030B00, but did not receive a response.  Rather port_id
0x030B00, sent a PLOGI of its own instead.  Thus, this PLOGI failure message
indicates that this HBA’s port sent PLOGI was canceled.

In fact, 0x030B00 is an initiator port and not a target port.  So, this is
expected behavior:

[   10.484676] lpfc 0000:27:00.0: 0:(0):0103 PRLI completes to NPort x030b00
Data: x0 x41000 x7 x1 x0
[   10.484678] lpfc 0000:27:00.0: 0:(0):0211 DSM in event x7 on NPort x30b00 in
state 4 rpi xd Data: x80080000 x30000
[   10.484680] lpfc 0000:27:00.0: 0:(0):6028 FCP NPR PRLI Cmpl Init 1 Target 0
EIP 0 AccCode x1


>   1. Although there're those failures, we can read/write date to SAN disks.
Good, because the lpfc driver logs indicate successful SCSI-PRLIs and eventual
successful SCSI discovery:

[   12.464025] lpfc 0000:27:00.0: 0:(0):0103 PRLI completes to NPort x051500
Data: x0 x41000 x7 x0 x0
[   12.464027] lpfc 0000:27:00.0: 0:(0):0211 DSM in event x7 on NPort x51500 in
state 4 rpi x18 Data: x80000000 x30008
[   12.464661] lpfc 0000:27:00.0: 0:(0):0212 DSM out state 7 on NPort x51500
rpi x18 Data: x80000000 x30008
[   12.466486] scsi 14:0:0:0: Direct-Access     LENOVO   DE_Series        0881
PQ: 1 ANSI: 5

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.

[Bug 219128] lpfc driver reports failed messages

[bugzilla-daemon]

https://bugzilla.kernel.org/show_bug.cgi?id=219128

Xavier (lixc17@lenovo.com) changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Assignee|linux-scsi@vger.kernel.org  |justin.tee@broadcom.com

-- 
You may reply to this email to add a comment.

You are receiving this mail because:
You are the assignee for the bug.