Broadcom 9460 raid card takes too long at system resuming

Broadcom 9460 raid card takes too long at system resuming

[Chris Chiu]

Hi,
    We found that the Broadcom 9460 RAID card will take ~40 seconds in
 megasas_resume. It is mainly waiting for the FW to come to ready
state, please refer to the following kernel log. The FW version is
"megasas: 07.714.04.00-rc1". It seems that the
megasas_transition_to_ready() loop costs ~40 seconds in
megasas_resume. However, the same megasas_transition_to_ready()
function only takes a few milliseconds to complete in
megasas_init_fw(). The .read_fw_status_reg maps to
megasas_read_fw_status_reg_fusion. I tried to add
pci_enable_device_mem() and pci_set_master before
megasas_transition_to_ready() in megasas_resume() but it makes no
difference.

I don't really know what makes the difference between driver probe and
resume. The lspci information of the raid controller is here
https://gist.github.com/mschiu77/e74ec084cc925643add845fa4dc31ab6. Any
suggestions about what I can do to find out the cause? Thanks.

[   62.357688] megaraid_sas 0000:45:00.0: megasas_resume is called
[   62.357719] megaraid_sas 0000:45:00.0: Waiting for FW to come to ready state
[  104.382571] megaraid_sas 0000:45:00.0: FW now in Ready state
[  104.382576] megaraid_sas 0000:45:00.0: 63 bit DMA mask and 63 bit
consistent mask
[  104.383350] megaraid_sas 0000:45:00.0: requested/available msix 33/33
[  104.383669] megaraid_sas 0000:45:00.0: Performance mode :Latency
[  104.383671] megaraid_sas 0000:45:00.0: FW supports sync cache        : Yes
[  104.383677] megaraid_sas 0000:45:00.0: megasas_disable_intr_fusion
is called outbound_intr_mask:0x40000009
[  104.550570] megaraid_sas 0000:45:00.0: FW provided
supportMaxExtLDs: 1       max_lds: 64
[  104.550574] megaraid_sas 0000:45:00.0: controller type       : MR(4096MB)
[  104.550575] megaraid_sas 0000:45:00.0: Online Controller Reset(OCR)
 : Enabled
[  104.550577] megaraid_sas 0000:45:00.0: Secure JBOD support   : Yes
[  104.550579] megaraid_sas 0000:45:00.0: NVMe passthru support : Yes
[  104.550581] megaraid_sas 0000:45:00.0: FW provided TM
TaskAbort/Reset timeout        : 6 secs/60 secs
[  104.550583] megaraid_sas 0000:45:00.0: JBOD sequence map support     : Yes
[  104.550585] megaraid_sas 0000:45:00.0: PCI Lane Margining support    : No
[  104.550999] megaraid_sas 0000:45:00.0: megasas_enable_intr_fusion
is called outbound_intr_mask:0x40000000

Chris

RE: Broadcom 9460 raid card takes too long at system resuming

[Kashyap Desai]

[-- Attachment #1: Type: text/plain, Size: 3218 bytes --]

Hi Chris -

Most likely behavior you explained is associated with how much time FW takes
to be activated.
In case of actual init from fresh boot, FW is already started running once
system is powered on, and user may not be aware of it.
By the time OS boot reach driver load from fresh boot, there was enough time
spend in system bring up.This is not true in case of resume (Hibernation.).

Kashyap

> -----Original Message-----
> From: Chris Chiu [mailto:chris.chiu@canonical.com]
> Sent: Monday, April 19, 2021 3:45 PM
> To: kashyap.desai@broadcom.com; sumit.saxena@broadcom.com;
> shivasharan.srikanteshwara@broadcom.com;
> megaraidlinux.pdl@broadcom.com
> Cc: linux-scsi@vger.kernel.org; Linux Kernel
> <linux-kernel@vger.kernel.org>
> Subject: Broadcom 9460 raid card takes too long at system resuming
>
> Hi,
>     We found that the Broadcom 9460 RAID card will take ~40 seconds in
> megasas_resume. It is mainly waiting for the FW to come to ready state,
> please refer to the following kernel log. The FW version is
> "megasas: 07.714.04.00-rc1". It seems that the
> megasas_transition_to_ready() loop costs ~40 seconds in megasas_resume.
> However, the same megasas_transition_to_ready() function only takes a few
> milliseconds to complete in megasas_init_fw(). The .read_fw_status_reg
> maps
> to megasas_read_fw_status_reg_fusion. I tried to add
> pci_enable_device_mem() and pci_set_master before
> megasas_transition_to_ready() in megasas_resume() but it makes no
> difference.
>
> I don't really know what makes the difference between driver probe and
> resume. The lspci information of the raid controller is here
> https://gist.github.com/mschiu77/e74ec084cc925643add845fa4dc31ab6.
> Any suggestions about what I can do to find out the cause? Thanks.
>
> [   62.357688] megaraid_sas 0000:45:00.0: megasas_resume is called
> [   62.357719] megaraid_sas 0000:45:00.0: Waiting for FW to come to ready
> state
> [  104.382571] megaraid_sas 0000:45:00.0: FW now in Ready state [
> 104.382576] megaraid_sas 0000:45:00.0: 63 bit DMA mask and 63 bit
> consistent mask [  104.383350] megaraid_sas 0000:45:00.0:
> requested/available msix 33/33 [  104.383669] megaraid_sas 0000:45:00.0:
> Performance mode :Latency
> [  104.383671] megaraid_sas 0000:45:00.0: FW supports sync cache        :
> Yes
> [  104.383677] megaraid_sas 0000:45:00.0: megasas_disable_intr_fusion is
> called outbound_intr_mask:0x40000009 [  104.550570] megaraid_sas
> 0000:45:00.0: FW provided
> supportMaxExtLDs: 1       max_lds: 64
> [  104.550574] megaraid_sas 0000:45:00.0: controller type       :
> MR(4096MB)
> [  104.550575] megaraid_sas 0000:45:00.0: Online Controller Reset(OCR)
>  : Enabled
> [  104.550577] megaraid_sas 0000:45:00.0: Secure JBOD support   : Yes
> [  104.550579] megaraid_sas 0000:45:00.0: NVMe passthru support : Yes [
> 104.550581] megaraid_sas 0000:45:00.0: FW provided TM
> TaskAbort/Reset timeout        : 6 secs/60 secs
> [  104.550583] megaraid_sas 0000:45:00.0: JBOD sequence map support     :
> Yes
> [  104.550585] megaraid_sas 0000:45:00.0: PCI Lane Margining support    :
> No
> [  104.550999] megaraid_sas 0000:45:00.0: megasas_enable_intr_fusion is
> called outbound_intr_mask:0x40000000
>
> Chris

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4212 bytes --]

Re: Broadcom 9460 raid card takes too long at system resuming

[Chris Chiu]

On Tue, Apr 20, 2021 at 9:21 PM Kashyap Desai
<kashyap.desai@broadcom.com> wrote:
>
> Hi Chris -
>
> Most likely behavior you explained is associated with how much time FW takes
> to be activated.
> In case of actual init from fresh boot, FW is already started running once
> system is powered on, and user may not be aware of it.
> By the time OS boot reach driver load from fresh boot, there was enough time
> spend in system bring up.This is not true in case of resume (Hibernation.).
>
> Kashyap
>
Thanks for the prompt response.

Chris

> > -----Original Message-----
> > From: Chris Chiu [mailto:chris.chiu@canonical.com]
> > Sent: Monday, April 19, 2021 3:45 PM
> > To: kashyap.desai@broadcom.com; sumit.saxena@broadcom.com;
> > shivasharan.srikanteshwara@broadcom.com;
> > megaraidlinux.pdl@broadcom.com
> > Cc: linux-scsi@vger.kernel.org; Linux Kernel
> > <linux-kernel@vger.kernel.org>
> > Subject: Broadcom 9460 raid card takes too long at system resuming
> >
> > Hi,
> >     We found that the Broadcom 9460 RAID card will take ~40 seconds in
> > megasas_resume. It is mainly waiting for the FW to come to ready state,
> > please refer to the following kernel log. The FW version is
> > "megasas: 07.714.04.00-rc1". It seems that the
> > megasas_transition_to_ready() loop costs ~40 seconds in megasas_resume.
> > However, the same megasas_transition_to_ready() function only takes a few
> > milliseconds to complete in megasas_init_fw(). The .read_fw_status_reg
> > maps
> > to megasas_read_fw_status_reg_fusion. I tried to add
> > pci_enable_device_mem() and pci_set_master before
> > megasas_transition_to_ready() in megasas_resume() but it makes no
> > difference.
> >
> > I don't really know what makes the difference between driver probe and
> > resume. The lspci information of the raid controller is here
> > https://gist.github.com/mschiu77/e74ec084cc925643add845fa4dc31ab6.
> > Any suggestions about what I can do to find out the cause? Thanks.
> >
> > [   62.357688] megaraid_sas 0000:45:00.0: megasas_resume is called
> > [   62.357719] megaraid_sas 0000:45:00.0: Waiting for FW to come to ready
> > state
> > [  104.382571] megaraid_sas 0000:45:00.0: FW now in Ready state [
> > 104.382576] megaraid_sas 0000:45:00.0: 63 bit DMA mask and 63 bit
> > consistent mask [  104.383350] megaraid_sas 0000:45:00.0:
> > requested/available msix 33/33 [  104.383669] megaraid_sas 0000:45:00.0:
> > Performance mode :Latency
> > [  104.383671] megaraid_sas 0000:45:00.0: FW supports sync cache        :
> > Yes
> > [  104.383677] megaraid_sas 0000:45:00.0: megasas_disable_intr_fusion is
> > called outbound_intr_mask:0x40000009 [  104.550570] megaraid_sas
> > 0000:45:00.0: FW provided
> > supportMaxExtLDs: 1       max_lds: 64
> > [  104.550574] megaraid_sas 0000:45:00.0: controller type       :
> > MR(4096MB)
> > [  104.550575] megaraid_sas 0000:45:00.0: Online Controller Reset(OCR)
> >  : Enabled
> > [  104.550577] megaraid_sas 0000:45:00.0: Secure JBOD support   : Yes
> > [  104.550579] megaraid_sas 0000:45:00.0: NVMe passthru support : Yes [
> > 104.550581] megaraid_sas 0000:45:00.0: FW provided TM
> > TaskAbort/Reset timeout        : 6 secs/60 secs
> > [  104.550583] megaraid_sas 0000:45:00.0: JBOD sequence map support     :
> > Yes
> > [  104.550585] megaraid_sas 0000:45:00.0: PCI Lane Margining support    :
> > No
> > [  104.550999] megaraid_sas 0000:45:00.0: megasas_enable_intr_fusion is
> > called outbound_intr_mask:0x40000000
> >
> > Chris