* [PATCH 0/4] pm80xx updates
@ 2020-09-25 6:16 Viswas G
2020-09-25 6:13 ` Jinpu Wang
` (4 more replies)
0 siblings, 5 replies; 16+ messages in thread
From: Viswas G @ 2020-09-25 6:16 UTC (permalink / raw)
To: linux-scsi
Cc: Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang
From: Viswas G <Viswas.G@microchip.com>
This patch set includes some enhancements in pm80xx driver.
Viswas G (4):
pm80xx : Increase number of supported queues.
pm80xx : Remove DMA memory allocation for ccb and device structures.
pm80xx : Increase the number of outstanding IO supported to 1024.
pm80xx : Driver version update
drivers/scsi/pm8001/pm8001_ctl.c | 6 +-
drivers/scsi/pm8001/pm8001_defs.h | 27 +++--
drivers/scsi/pm8001/pm8001_hwi.c | 38 +++----
drivers/scsi/pm8001/pm8001_init.c | 220 ++++++++++++++++++++++++++------------
drivers/scsi/pm8001/pm8001_sas.h | 15 ++-
drivers/scsi/pm8001/pm80xx_hwi.c | 109 ++++++++++---------
6 files changed, 253 insertions(+), 162 deletions(-)
--
2.16.3
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [PATCH 0/4] pm80xx updates 2020-09-25 6:16 [PATCH 0/4] pm80xx updates Viswas G @ 2020-09-25 6:13 ` Jinpu Wang 2020-09-25 6:36 ` Viswas.G 2020-09-25 6:16 ` [PATCH 1/4] pm80xx : Increase number of supported queues Viswas G ` (3 subsequent siblings) 4 siblings, 1 reply; 16+ messages in thread From: Jinpu Wang @ 2020-09-25 6:13 UTC (permalink / raw) To: Viswas G Cc: Linux SCSI Mailinglist, Vasanthalakshmi.Tharmarajan, Viswas G, Ruksar.devadi Hi Viswas, On Fri, Sep 25, 2020 at 8:06 AM Viswas G <Viswas.G@microchip.com.com> wrote: > > From: Viswas G <Viswas.G@microchip.com> > > This patch set includes some enhancements in pm80xx driver. Thanks for the patches. the first and 3rd one, seems to be performance improvement, can you share your performance number? > > Viswas G (4): > pm80xx : Increase number of supported queues. > pm80xx : Remove DMA memory allocation for ccb and device structures. > pm80xx : Increase the number of outstanding IO supported to 1024. > pm80xx : Driver version update Thanks > > drivers/scsi/pm8001/pm8001_ctl.c | 6 +- > drivers/scsi/pm8001/pm8001_defs.h | 27 +++-- > drivers/scsi/pm8001/pm8001_hwi.c | 38 +++---- > drivers/scsi/pm8001/pm8001_init.c | 220 ++++++++++++++++++++++++++------------ > drivers/scsi/pm8001/pm8001_sas.h | 15 ++- > drivers/scsi/pm8001/pm80xx_hwi.c | 109 ++++++++++--------- > 6 files changed, 253 insertions(+), 162 deletions(-) > > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH 0/4] pm80xx updates 2020-09-25 6:13 ` Jinpu Wang @ 2020-09-25 6:36 ` Viswas.G 0 siblings, 0 replies; 16+ messages in thread From: Viswas.G @ 2020-09-25 6:36 UTC (permalink / raw) To: jinpu.wang, Viswas.G Cc: linux-scsi, Vasanthalakshmi.Tharmarajan, Ruksar.devadi I could see below results in my test setup. fio_sqrd_bs512_qd.txt: read: IOPS=1000k, BW=488MiB/s (512MB/s)(28.7GiB/60123msec) fio_sqrd_bs64k_qd.txt: read: IOPS=37.0k, BW=2377MiB/s (2493MB/s)(139GiB/60000msec) fio_sqw_bs256k_qd.txt: write: IOPS=4310, BW=1102MiB/s (1156MB/s)(66.2GiB/61531msec); 0 zone resets fio_sqw_bs4k_qd.txt: write: IOPS=216k, BW=846MiB/s (888MB/s)(47.3GiB/57189msec); 0 zone resets With patches: fio_sqrd_bs512_qd.txt: read: IOPS=1539k, BW=752MiB/s (789MB/s)(44.2GiB/60174msec) => +53.9% fio_sqrd_bs64k_qd.txt: read: IOPS=30.9k, BW=1946MiB/s (2041MB/s)(114GiB/60000msec) fio_sqw_bs256k_qd.txt: write: IOPS=4309, BW=1102MiB/s (1155MB/s)(66.3GiB/61602msec); 0 zone resets fio_sqw_bs4k_qd.txt: write: IOPS=300k, BW=1174MiB/s (1231MB/s)(68.0GiB/59355msec); 0 zone resets => +38% Regards, Viswas G > -----Original Message----- > From: Jinpu Wang <jinpu.wang@cloud.ionos.com> > Sent: Friday, September 25, 2020 11:43 AM > To: Viswas G <Viswas.G@microchip.com.com> > Cc: Linux SCSI Mailinglist <linux-scsi@vger.kernel.org>; Vasanthalakshmi > Tharmarajan - I30664 <Vasanthalakshmi.Tharmarajan@microchip.com>; > Viswas G - I30667 <Viswas.G@microchip.com>; Ruksar Devadi - I52327 > <Ruksar.devadi@microchip.com> > Subject: Re: [PATCH 0/4] pm80xx updates > > EXTERNAL EMAIL: Do not click links or open attachments unless you know > the content is safe > > Hi Viswas, > On Fri, Sep 25, 2020 at 8:06 AM Viswas G <Viswas.G@microchip.com.com> > wrote: > > > > From: Viswas G <Viswas.G@microchip.com> > > > > This patch set includes some enhancements in pm80xx driver. > Thanks for the patches. the first and 3rd one, seems to be performance > improvement, can you share your performance number? > > > > > > Viswas G (4): > > pm80xx : Increase number of supported queues. > > pm80xx : Remove DMA memory allocation for ccb and device structures. > > pm80xx : Increase the number of outstanding IO supported to 1024. > > pm80xx : Driver version update > Thanks > > > > drivers/scsi/pm8001/pm8001_ctl.c | 6 +- > > drivers/scsi/pm8001/pm8001_defs.h | 27 +++-- > > drivers/scsi/pm8001/pm8001_hwi.c | 38 +++---- > > drivers/scsi/pm8001/pm8001_init.c | 220 > > ++++++++++++++++++++++++++------------ > > drivers/scsi/pm8001/pm8001_sas.h | 15 ++- > > drivers/scsi/pm8001/pm80xx_hwi.c | 109 ++++++++++--------- > > 6 files changed, 253 insertions(+), 162 deletions(-) > > > > -- > > 2.16.3 > > ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH 1/4] pm80xx : Increase number of supported queues. 2020-09-25 6:16 [PATCH 0/4] pm80xx updates Viswas G 2020-09-25 6:13 ` Jinpu Wang @ 2020-09-25 6:16 ` Viswas G 2020-09-25 9:14 ` John Garry 2020-10-05 11:13 ` Jinpu Wang 2020-09-25 6:16 ` [PATCH 2/4] pm80xx : Remove DMA memory allocation for ccb and device Viswas G ` (2 subsequent siblings) 4 siblings, 2 replies; 16+ messages in thread From: Viswas G @ 2020-09-25 6:16 UTC (permalink / raw) To: linux-scsi Cc: Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang From: Viswas G <Viswas.G@microchip.com> The number of supported Inbound and outbound queues is increased to 64, and the number of queues used are decided based on number of CPU cores online and number of msix allocated. Added locks per queue instead of global lock. Signed-off-by: Viswas G <Viswas.G@microchip.com> Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> --- drivers/scsi/pm8001/pm8001_ctl.c | 6 +- drivers/scsi/pm8001/pm8001_defs.h | 17 +++--- drivers/scsi/pm8001/pm8001_hwi.c | 32 ++++++----- drivers/scsi/pm8001/pm8001_init.c | 117 ++++++++++++++++++++++++-------------- drivers/scsi/pm8001/pm8001_sas.h | 11 +++- drivers/scsi/pm8001/pm80xx_hwi.c | 81 +++++++++++++++----------- 6 files changed, 160 insertions(+), 104 deletions(-) diff --git a/drivers/scsi/pm8001/pm8001_ctl.c b/drivers/scsi/pm8001/pm8001_ctl.c index 77c805db2724..3587f7c8a428 100644 --- a/drivers/scsi/pm8001/pm8001_ctl.c +++ b/drivers/scsi/pm8001/pm8001_ctl.c @@ -408,9 +408,10 @@ static ssize_t pm8001_ctl_ib_queue_log_show(struct device *cdev, int offset; char *str = buf; int start = 0; + u32 ib_offset = pm8001_ha->ib_offset; #define IB_MEMMAP(c) \ (*(u32 *)((u8 *)pm8001_ha-> \ - memoryMap.region[IB].virt_ptr + \ + memoryMap.region[ib_offset].virt_ptr + \ pm8001_ha->evtlog_ib_offset + (c))) for (offset = 0; offset < IB_OB_READ_TIMES; offset++) { @@ -442,9 +443,10 @@ static ssize_t pm8001_ctl_ob_queue_log_show(struct device *cdev, int offset; char *str = buf; int start = 0; + u32 ob_offset = pm8001_ha->ob_offset; #define OB_MEMMAP(c) \ (*(u32 *)((u8 *)pm8001_ha-> \ - memoryMap.region[OB].virt_ptr + \ + memoryMap.region[ob_offset].virt_ptr + \ pm8001_ha->evtlog_ob_offset + (c))) for (offset = 0; offset < IB_OB_READ_TIMES; offset++) { diff --git a/drivers/scsi/pm8001/pm8001_defs.h b/drivers/scsi/pm8001/pm8001_defs.h index 1c7f15fd69ce..a4f52a5a449e 100644 --- a/drivers/scsi/pm8001/pm8001_defs.h +++ b/drivers/scsi/pm8001/pm8001_defs.h @@ -77,10 +77,8 @@ enum port_type { /* driver compile-time configuration */ #define PM8001_MAX_CCB 256 /* max ccbs supported */ #define PM8001_MPI_QUEUE 1024 /* maximum mpi queue entries */ -#define PM8001_MAX_INB_NUM 1 -#define PM8001_MAX_OUTB_NUM 1 -#define PM8001_MAX_SPCV_INB_NUM 1 -#define PM8001_MAX_SPCV_OUTB_NUM 4 +#define PM8001_MAX_INB_NUM 64 +#define PM8001_MAX_OUTB_NUM 64 #define PM8001_CAN_QUEUE 508 /* SCSI Queue depth */ /* Inbound/Outbound queue size */ @@ -94,11 +92,6 @@ enum port_type { #define PM8001_MAX_MSIX_VEC 64 /* max msi-x int for spcv/ve */ #define USI_MAX_MEMCNT_BASE 5 -#define IB (USI_MAX_MEMCNT_BASE + 1) -#define CI (IB + PM8001_MAX_SPCV_INB_NUM) -#define OB (CI + PM8001_MAX_SPCV_INB_NUM) -#define PI (OB + PM8001_MAX_SPCV_OUTB_NUM) -#define USI_MAX_MEMCNT (PI + PM8001_MAX_SPCV_OUTB_NUM) #define CONFIG_SCSI_PM8001_MAX_DMA_SG 528 #define PM8001_MAX_DMA_SG CONFIG_SCSI_PM8001_MAX_DMA_SG enum memory_region_num { @@ -112,6 +105,12 @@ enum memory_region_num { }; #define PM8001_EVENT_LOG_SIZE (128 * 1024) +/** + * maximum DMA memory regions(number of IBQ + number of IBQ CI + * + number of OBQ + number of OBQ PI) + */ +#define USI_MAX_MEMCNT (USI_MAX_MEMCNT_BASE + 1 + ((2 * PM8001_MAX_INB_NUM) \ + + (2 * PM8001_MAX_OUTB_NUM))) /*error code*/ enum mpi_err { MPI_IO_STATUS_SUCCESS = 0x0, diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c index e9a939230b15..e9106575a30f 100644 --- a/drivers/scsi/pm8001/pm8001_hwi.c +++ b/drivers/scsi/pm8001/pm8001_hwi.c @@ -189,6 +189,10 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) u32 offsetib, offsetob; void __iomem *addressib = pm8001_ha->inbnd_q_tbl_addr; void __iomem *addressob = pm8001_ha->outbnd_q_tbl_addr; + u32 ib_offset = pm8001_ha->ib_offset; + u32 ob_offset = pm8001_ha->ob_offset; + u32 ci_offset = pm8001_ha->ci_offset; + u32 pi_offset = pm8001_ha->pi_offset; pm8001_ha->main_cfg_tbl.pm8001_tbl.inbound_q_nppd_hppd = 0; pm8001_ha->main_cfg_tbl.pm8001_tbl.outbound_hw_event_pid0_3 = 0; @@ -223,19 +227,19 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) pm8001_ha->inbnd_q_tbl[i].element_pri_size_cnt = PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x00<<30); pm8001_ha->inbnd_q_tbl[i].upper_base_addr = - pm8001_ha->memoryMap.region[IB + i].phys_addr_hi; + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_hi; pm8001_ha->inbnd_q_tbl[i].lower_base_addr = - pm8001_ha->memoryMap.region[IB + i].phys_addr_lo; + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_lo; pm8001_ha->inbnd_q_tbl[i].base_virt = - (u8 *)pm8001_ha->memoryMap.region[IB + i].virt_ptr; + (u8 *)pm8001_ha->memoryMap.region[ib_offset + i].virt_ptr; pm8001_ha->inbnd_q_tbl[i].total_length = - pm8001_ha->memoryMap.region[IB + i].total_len; + pm8001_ha->memoryMap.region[ib_offset + i].total_len; pm8001_ha->inbnd_q_tbl[i].ci_upper_base_addr = - pm8001_ha->memoryMap.region[CI + i].phys_addr_hi; + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_hi; pm8001_ha->inbnd_q_tbl[i].ci_lower_base_addr = - pm8001_ha->memoryMap.region[CI + i].phys_addr_lo; + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_lo; pm8001_ha->inbnd_q_tbl[i].ci_virt = - pm8001_ha->memoryMap.region[CI + i].virt_ptr; + pm8001_ha->memoryMap.region[ci_offset + i].virt_ptr; offsetib = i * 0x20; pm8001_ha->inbnd_q_tbl[i].pi_pci_bar = get_pci_bar_index(pm8001_mr32(addressib, @@ -249,21 +253,21 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) pm8001_ha->outbnd_q_tbl[i].element_size_cnt = PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x01<<30); pm8001_ha->outbnd_q_tbl[i].upper_base_addr = - pm8001_ha->memoryMap.region[OB + i].phys_addr_hi; + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_hi; pm8001_ha->outbnd_q_tbl[i].lower_base_addr = - pm8001_ha->memoryMap.region[OB + i].phys_addr_lo; + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_lo; pm8001_ha->outbnd_q_tbl[i].base_virt = - (u8 *)pm8001_ha->memoryMap.region[OB + i].virt_ptr; + (u8 *)pm8001_ha->memoryMap.region[ob_offset + i].virt_ptr; pm8001_ha->outbnd_q_tbl[i].total_length = - pm8001_ha->memoryMap.region[OB + i].total_len; + pm8001_ha->memoryMap.region[ob_offset + i].total_len; pm8001_ha->outbnd_q_tbl[i].pi_upper_base_addr = - pm8001_ha->memoryMap.region[PI + i].phys_addr_hi; + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_hi; pm8001_ha->outbnd_q_tbl[i].pi_lower_base_addr = - pm8001_ha->memoryMap.region[PI + i].phys_addr_lo; + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_lo; pm8001_ha->outbnd_q_tbl[i].interrup_vec_cnt_delay = 0 | (10 << 16) | (i << 24); pm8001_ha->outbnd_q_tbl[i].pi_virt = - pm8001_ha->memoryMap.region[PI + i].virt_ptr; + pm8001_ha->memoryMap.region[pi_offset + i].virt_ptr; offsetob = i * 0x24; pm8001_ha->outbnd_q_tbl[i].ci_pci_bar = get_pci_bar_index(pm8001_mr32(addressob, diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c index 20fa96cbc9d3..c744f846e08d 100644 --- a/drivers/scsi/pm8001/pm8001_init.c +++ b/drivers/scsi/pm8001/pm8001_init.c @@ -264,12 +264,36 @@ static u32 pm8001_request_irq(struct pm8001_hba_info *pm8001_ha); static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, const struct pci_device_id *ent) { - int i; + int i, count = 0, rc = 0; + u32 ci_offset, ib_offset, ob_offset, pi_offset; + struct inbound_queue_table *circularQ; + spin_lock_init(&pm8001_ha->lock); spin_lock_init(&pm8001_ha->bitmap_lock); PM8001_INIT_DBG(pm8001_ha, pm8001_printk("pm8001_alloc: PHY:%x\n", pm8001_ha->chip->n_phy)); + + /* Setup Interrupt */ + rc = pm8001_setup_irq(pm8001_ha); + if (rc) { + PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( + "pm8001_setup_irq failed [ret: %d]\n", rc)); + goto err_out_shost; + } + /* Request Interrupt */ + rc = pm8001_request_irq(pm8001_ha); + if (rc) + goto err_out_shost; + + count = pm8001_ha->max_q_num; + /* Queues are chosen based on the number of cores/msix availability */ + ib_offset = pm8001_ha->ib_offset = USI_MAX_MEMCNT_BASE + 1; + ci_offset = pm8001_ha->ci_offset = ib_offset + count; + ob_offset = pm8001_ha->ob_offset = ci_offset + count; + pi_offset = pm8001_ha->pi_offset = ob_offset + count; + pm8001_ha->max_memcnt = pi_offset + count; + for (i = 0; i < pm8001_ha->chip->n_phy; i++) { pm8001_phy_init(pm8001_ha, i); pm8001_ha->port[i].wide_port_phymap = 0; @@ -293,54 +317,62 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, pm8001_ha->memoryMap.region[IOP].total_len = PM8001_EVENT_LOG_SIZE; pm8001_ha->memoryMap.region[IOP].alignment = 32; - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) { + for (i = 0; i < count; i++) { + circularQ = &pm8001_ha->inbnd_q_tbl[i]; + spin_lock_init(&circularQ->iq_lock); /* MPI Memory region 3 for consumer Index of inbound queues */ - pm8001_ha->memoryMap.region[CI+i].num_elements = 1; - pm8001_ha->memoryMap.region[CI+i].element_size = 4; - pm8001_ha->memoryMap.region[CI+i].total_len = 4; - pm8001_ha->memoryMap.region[CI+i].alignment = 4; + pm8001_ha->memoryMap.region[ci_offset+i].num_elements = 1; + pm8001_ha->memoryMap.region[ci_offset+i].element_size = 4; + pm8001_ha->memoryMap.region[ci_offset+i].total_len = 4; + pm8001_ha->memoryMap.region[ci_offset+i].alignment = 4; if ((ent->driver_data) != chip_8001) { /* MPI Memory region 5 inbound queues */ - pm8001_ha->memoryMap.region[IB+i].num_elements = + pm8001_ha->memoryMap.region[ib_offset+i].num_elements = PM8001_MPI_QUEUE; - pm8001_ha->memoryMap.region[IB+i].element_size = 128; - pm8001_ha->memoryMap.region[IB+i].total_len = + pm8001_ha->memoryMap.region[ib_offset+i].element_size + = 128; + pm8001_ha->memoryMap.region[ib_offset+i].total_len = PM8001_MPI_QUEUE * 128; - pm8001_ha->memoryMap.region[IB+i].alignment = 128; + pm8001_ha->memoryMap.region[ib_offset+i].alignment + = 128; } else { - pm8001_ha->memoryMap.region[IB+i].num_elements = + pm8001_ha->memoryMap.region[ib_offset+i].num_elements = PM8001_MPI_QUEUE; - pm8001_ha->memoryMap.region[IB+i].element_size = 64; - pm8001_ha->memoryMap.region[IB+i].total_len = + pm8001_ha->memoryMap.region[ib_offset+i].element_size + = 64; + pm8001_ha->memoryMap.region[ib_offset+i].total_len = PM8001_MPI_QUEUE * 64; - pm8001_ha->memoryMap.region[IB+i].alignment = 64; + pm8001_ha->memoryMap.region[ib_offset+i].alignment = 64; } } - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) { + for (i = 0; i < count; i++) { /* MPI Memory region 4 for producer Index of outbound queues */ - pm8001_ha->memoryMap.region[PI+i].num_elements = 1; - pm8001_ha->memoryMap.region[PI+i].element_size = 4; - pm8001_ha->memoryMap.region[PI+i].total_len = 4; - pm8001_ha->memoryMap.region[PI+i].alignment = 4; + pm8001_ha->memoryMap.region[pi_offset+i].num_elements = 1; + pm8001_ha->memoryMap.region[pi_offset+i].element_size = 4; + pm8001_ha->memoryMap.region[pi_offset+i].total_len = 4; + pm8001_ha->memoryMap.region[pi_offset+i].alignment = 4; if (ent->driver_data != chip_8001) { /* MPI Memory region 6 Outbound queues */ - pm8001_ha->memoryMap.region[OB+i].num_elements = + pm8001_ha->memoryMap.region[ob_offset+i].num_elements = PM8001_MPI_QUEUE; - pm8001_ha->memoryMap.region[OB+i].element_size = 128; - pm8001_ha->memoryMap.region[OB+i].total_len = + pm8001_ha->memoryMap.region[ob_offset+i].element_size + = 128; + pm8001_ha->memoryMap.region[ob_offset+i].total_len = PM8001_MPI_QUEUE * 128; - pm8001_ha->memoryMap.region[OB+i].alignment = 128; + pm8001_ha->memoryMap.region[ob_offset+i].alignment + = 128; } else { /* MPI Memory region 6 Outbound queues */ - pm8001_ha->memoryMap.region[OB+i].num_elements = + pm8001_ha->memoryMap.region[ob_offset+i].num_elements = PM8001_MPI_QUEUE; - pm8001_ha->memoryMap.region[OB+i].element_size = 64; - pm8001_ha->memoryMap.region[OB+i].total_len = + pm8001_ha->memoryMap.region[ob_offset+i].element_size + = 64; + pm8001_ha->memoryMap.region[ob_offset+i].total_len = PM8001_MPI_QUEUE * 64; - pm8001_ha->memoryMap.region[OB+i].alignment = 64; + pm8001_ha->memoryMap.region[ob_offset+i].alignment = 64; } } @@ -369,7 +401,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, pm8001_ha->memoryMap.region[FORENSIC_MEM].total_len = 0x10000; pm8001_ha->memoryMap.region[FORENSIC_MEM].element_size = 0x10000; pm8001_ha->memoryMap.region[FORENSIC_MEM].alignment = 0x10000; - for (i = 0; i < USI_MAX_MEMCNT; i++) { + for (i = 0; i < pm8001_ha->max_memcnt; i++) { if (pm8001_mem_alloc(pm8001_ha->pdev, &pm8001_ha->memoryMap.region[i].virt_ptr, &pm8001_ha->memoryMap.region[i].phys_addr, @@ -405,6 +437,8 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, /* Initialize tags */ pm8001_tag_init(pm8001_ha); return 0; +err_out_shost: + scsi_remove_host(pm8001_ha->shost); err_out: return 1; } @@ -899,7 +933,8 @@ static int pm8001_configure_phy_settings(struct pm8001_hba_info *pm8001_ha) static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) { u32 number_of_intr; - int rc; + int rc, cpu_online_count; + unsigned int allocated_irq_vectors; /* SPCv controllers supports 64 msi-x */ if (pm8001_ha->chip_id == chip_8001) { @@ -908,13 +943,21 @@ static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) number_of_intr = PM8001_MAX_MSIX_VEC; } + cpu_online_count = num_online_cpus(); + number_of_intr = min_t(int, cpu_online_count, number_of_intr); rc = pci_alloc_irq_vectors(pm8001_ha->pdev, number_of_intr, number_of_intr, PCI_IRQ_MSIX); - number_of_intr = rc; + allocated_irq_vectors = rc; if (rc < 0) return rc; + + /*Assigns the number of interrupts */ + number_of_intr = min_t(int, allocated_irq_vectors, number_of_intr); pm8001_ha->number_of_intr = number_of_intr; + /*maximum queue number updating in HBA structure */ + pm8001_ha->max_q_num = number_of_intr; + PM8001_INIT_DBG(pm8001_ha, pm8001_printk( "pci_alloc_irq_vectors request ret:%d no of intr %d\n", rc, pm8001_ha->number_of_intr)); @@ -1069,13 +1112,6 @@ static int pm8001_pci_probe(struct pci_dev *pdev, rc = -ENOMEM; goto err_out_free; } - /* Setup Interrupt */ - rc = pm8001_setup_irq(pm8001_ha); - if (rc) { - PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( - "pm8001_setup_irq failed [ret: %d]\n", rc)); - goto err_out_shost; - } PM8001_CHIP_DISP->chip_soft_rst(pm8001_ha); rc = PM8001_CHIP_DISP->chip_init(pm8001_ha); @@ -1088,13 +1124,6 @@ static int pm8001_pci_probe(struct pci_dev *pdev, rc = scsi_add_host(shost, &pdev->dev); if (rc) goto err_out_ha_free; - /* Request Interrupt */ - rc = pm8001_request_irq(pm8001_ha); - if (rc) { - PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( - "pm8001_request_irq failed [ret: %d]\n", rc)); - goto err_out_shost; - } PM8001_CHIP_DISP->interrupt_enable(pm8001_ha, 0); if (pm8001_ha->chip_id != chip_8001) { diff --git a/drivers/scsi/pm8001/pm8001_sas.h b/drivers/scsi/pm8001/pm8001_sas.h index ae7ba9b3c4bc..bdfce3c3f619 100644 --- a/drivers/scsi/pm8001/pm8001_sas.h +++ b/drivers/scsi/pm8001/pm8001_sas.h @@ -468,6 +468,7 @@ struct inbound_queue_table { u32 reserved; __le32 consumer_index; u32 producer_idx; + spinlock_t iq_lock; }; struct outbound_queue_table { u32 element_size_cnt; @@ -524,8 +525,8 @@ struct pm8001_hba_info { void __iomem *fatal_tbl_addr; /*MPI IVT Table Addr */ union main_cfg_table main_cfg_tbl; union general_status_table gs_tbl; - struct inbound_queue_table inbnd_q_tbl[PM8001_MAX_SPCV_INB_NUM]; - struct outbound_queue_table outbnd_q_tbl[PM8001_MAX_SPCV_OUTB_NUM]; + struct inbound_queue_table inbnd_q_tbl[PM8001_MAX_INB_NUM]; + struct outbound_queue_table outbnd_q_tbl[PM8001_MAX_OUTB_NUM]; struct sas_phy_attribute_table phy_attr_table; /* MPI SAS PHY attributes */ u8 sas_addr[SAS_ADDR_SIZE]; @@ -561,6 +562,12 @@ struct pm8001_hba_info { u32 reset_in_progress; u32 non_fatal_count; u32 non_fatal_read_length; + u32 max_q_num; + u32 ib_offset; + u32 ob_offset; + u32 ci_offset; + u32 pi_offset; + u32 max_memcnt; }; struct pm8001_work { diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c index b42f41d1ed49..26e9e8877107 100644 --- a/drivers/scsi/pm8001/pm80xx_hwi.c +++ b/drivers/scsi/pm8001/pm80xx_hwi.c @@ -720,7 +720,7 @@ static void read_inbnd_queue_table(struct pm8001_hba_info *pm8001_ha) { int i; void __iomem *address = pm8001_ha->inbnd_q_tbl_addr; - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) { + for (i = 0; i < PM8001_MAX_INB_NUM; i++) { u32 offset = i * 0x20; pm8001_ha->inbnd_q_tbl[i].pi_pci_bar = get_pci_bar_index(pm8001_mr32(address, @@ -738,7 +738,7 @@ static void read_outbnd_queue_table(struct pm8001_hba_info *pm8001_ha) { int i; void __iomem *address = pm8001_ha->outbnd_q_tbl_addr; - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) { + for (i = 0; i < PM8001_MAX_OUTB_NUM; i++) { u32 offset = i * 0x24; pm8001_ha->outbnd_q_tbl[i].ci_pci_bar = get_pci_bar_index(pm8001_mr32(address, @@ -758,6 +758,10 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) u32 offsetib, offsetob; void __iomem *addressib = pm8001_ha->inbnd_q_tbl_addr; void __iomem *addressob = pm8001_ha->outbnd_q_tbl_addr; + u32 ib_offset = pm8001_ha->ib_offset; + u32 ob_offset = pm8001_ha->ob_offset; + u32 ci_offset = pm8001_ha->ci_offset; + u32 pi_offset = pm8001_ha->pi_offset; pm8001_ha->main_cfg_tbl.pm80xx_tbl.upper_event_log_addr = pm8001_ha->memoryMap.region[AAP1].phys_addr_hi; @@ -778,23 +782,23 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) /* Disable end to end CRC checking */ pm8001_ha->main_cfg_tbl.pm80xx_tbl.crc_core_dump = (0x1 << 16); - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) { + for (i = 0; i < pm8001_ha->max_q_num; i++) { pm8001_ha->inbnd_q_tbl[i].element_pri_size_cnt = PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x00<<30); pm8001_ha->inbnd_q_tbl[i].upper_base_addr = - pm8001_ha->memoryMap.region[IB + i].phys_addr_hi; + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_hi; pm8001_ha->inbnd_q_tbl[i].lower_base_addr = - pm8001_ha->memoryMap.region[IB + i].phys_addr_lo; + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_lo; pm8001_ha->inbnd_q_tbl[i].base_virt = - (u8 *)pm8001_ha->memoryMap.region[IB + i].virt_ptr; + (u8 *)pm8001_ha->memoryMap.region[ib_offset + i].virt_ptr; pm8001_ha->inbnd_q_tbl[i].total_length = - pm8001_ha->memoryMap.region[IB + i].total_len; + pm8001_ha->memoryMap.region[ib_offset + i].total_len; pm8001_ha->inbnd_q_tbl[i].ci_upper_base_addr = - pm8001_ha->memoryMap.region[CI + i].phys_addr_hi; + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_hi; pm8001_ha->inbnd_q_tbl[i].ci_lower_base_addr = - pm8001_ha->memoryMap.region[CI + i].phys_addr_lo; + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_lo; pm8001_ha->inbnd_q_tbl[i].ci_virt = - pm8001_ha->memoryMap.region[CI + i].virt_ptr; + pm8001_ha->memoryMap.region[ci_offset + i].virt_ptr; offsetib = i * 0x20; pm8001_ha->inbnd_q_tbl[i].pi_pci_bar = get_pci_bar_index(pm8001_mr32(addressib, @@ -809,25 +813,25 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) pm8001_ha->inbnd_q_tbl[i].pi_pci_bar, pm8001_ha->inbnd_q_tbl[i].pi_offset)); } - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) { + for (i = 0; i < pm8001_ha->max_q_num; i++) { pm8001_ha->outbnd_q_tbl[i].element_size_cnt = PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x01<<30); pm8001_ha->outbnd_q_tbl[i].upper_base_addr = - pm8001_ha->memoryMap.region[OB + i].phys_addr_hi; + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_hi; pm8001_ha->outbnd_q_tbl[i].lower_base_addr = - pm8001_ha->memoryMap.region[OB + i].phys_addr_lo; + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_lo; pm8001_ha->outbnd_q_tbl[i].base_virt = - (u8 *)pm8001_ha->memoryMap.region[OB + i].virt_ptr; + (u8 *)pm8001_ha->memoryMap.region[ob_offset + i].virt_ptr; pm8001_ha->outbnd_q_tbl[i].total_length = - pm8001_ha->memoryMap.region[OB + i].total_len; + pm8001_ha->memoryMap.region[ob_offset + i].total_len; pm8001_ha->outbnd_q_tbl[i].pi_upper_base_addr = - pm8001_ha->memoryMap.region[PI + i].phys_addr_hi; + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_hi; pm8001_ha->outbnd_q_tbl[i].pi_lower_base_addr = - pm8001_ha->memoryMap.region[PI + i].phys_addr_lo; + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_lo; /* interrupt vector based on oq */ pm8001_ha->outbnd_q_tbl[i].interrup_vec_cnt_delay = (i << 24); pm8001_ha->outbnd_q_tbl[i].pi_virt = - pm8001_ha->memoryMap.region[PI + i].virt_ptr; + pm8001_ha->memoryMap.region[pi_offset + i].virt_ptr; offsetob = i * 0x24; pm8001_ha->outbnd_q_tbl[i].ci_pci_bar = get_pci_bar_index(pm8001_mr32(addressob, @@ -871,7 +875,7 @@ static void update_main_config_table(struct pm8001_hba_info *pm8001_ha) pm8001_ha->main_cfg_tbl.pm80xx_tbl.pcs_event_log_severity); /* Update Fatal error interrupt vector */ pm8001_ha->main_cfg_tbl.pm80xx_tbl.fatal_err_interrupt |= - ((pm8001_ha->number_of_intr - 1) << 8); + ((pm8001_ha->max_q_num - 1) << 8); pm8001_mw32(address, MAIN_FATAL_ERROR_INTERRUPT, pm8001_ha->main_cfg_tbl.pm80xx_tbl.fatal_err_interrupt); PM8001_DEV_DBG(pm8001_ha, pm8001_printk( @@ -1010,8 +1014,12 @@ static int mpi_init_check(struct pm8001_hba_info *pm8001_ha) value &= SPCv_MSGU_CFG_TABLE_UPDATE; } while ((value != 0) && (--max_wait_count)); - if (!max_wait_count) - return -1; + if (!max_wait_count) { + /* additional check */ + PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( + "Inb doorbell clear not toggled[value:%x]\n", value)); + return -EBUSY; + } /* check the MPI-State for initialization upto 100ms*/ max_wait_count = 100 * 1000;/* 100 msec */ do { @@ -1022,12 +1030,12 @@ static int mpi_init_check(struct pm8001_hba_info *pm8001_ha) } while ((GST_MPI_STATE_INIT != (gst_len_mpistate & GST_MPI_STATE_MASK)) && (--max_wait_count)); if (!max_wait_count) - return -1; + return -EBUSY; /* check MPI Initialization error */ gst_len_mpistate = gst_len_mpistate >> 16; if (0x0000 != gst_len_mpistate) - return -1; + return -EBUSY; return 0; } @@ -1469,11 +1477,10 @@ static int pm80xx_chip_init(struct pm8001_hba_info *pm8001_ha) /* update main config table ,inbound table and outbound table */ update_main_config_table(pm8001_ha); - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) + for (i = 0; i < pm8001_ha->max_q_num; i++) { update_inbnd_queue_table(pm8001_ha, i); - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) update_outbnd_queue_table(pm8001_ha, i); - + } /* notify firmware update finished and check initialization status */ if (0 == mpi_init_check(pm8001_ha)) { PM8001_INIT_DBG(pm8001_ha, @@ -4191,7 +4198,7 @@ static int process_oq(struct pm8001_hba_info *pm8001_ha, u8 vec) unsigned long flags; u32 regval; - if (vec == (pm8001_ha->number_of_intr - 1)) { + if (vec == (pm8001_ha->max_q_num - 1)) { regval = pm8001_cr32(pm8001_ha, 0, MSGU_SCRATCH_PAD_1); if ((regval & SCRATCH_PAD_MIPSALL_READY) != SCRATCH_PAD_MIPSALL_READY) { @@ -4274,6 +4281,7 @@ static int pm80xx_chip_smp_req(struct pm8001_hba_info *pm8001_ha, char *preq_dma_addr = NULL; __le64 tmp_addr; u32 i, length; + unsigned long flags; memset(&smp_cmd, 0, sizeof(smp_cmd)); /* @@ -4369,8 +4377,10 @@ static int pm80xx_chip_smp_req(struct pm8001_hba_info *pm8001_ha, build_smp_cmd(pm8001_dev->device_id, smp_cmd.tag, &smp_cmd, pm8001_ha->smp_exp_mode, length); + spin_lock_irqsave(&circularQ->iq_lock, flags); rc = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &smp_cmd, sizeof(smp_cmd), 0); + spin_unlock_irqrestore(&circularQ->iq_lock, flags); if (rc) goto err_out_2; return 0; @@ -4434,7 +4444,8 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, u64 phys_addr, start_addr, end_addr; u32 end_addr_high, end_addr_low; struct inbound_queue_table *circularQ; - u32 q_index; + unsigned long flags; + u32 q_index, cpu_id; u32 opc = OPC_INB_SSPINIIOSTART; memset(&ssp_cmd, 0, sizeof(ssp_cmd)); memcpy(ssp_cmd.ssp_iu.lun, task->ssp_task.LUN, 8); @@ -4453,7 +4464,8 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, ssp_cmd.ssp_iu.efb_prio_attr |= (task->ssp_task.task_attr & 7); memcpy(ssp_cmd.ssp_iu.cdb, task->ssp_task.cmd->cmnd, task->ssp_task.cmd->cmd_len); - q_index = (u32) (pm8001_dev->id & 0x00ffffff) % PM8001_MAX_INB_NUM; + cpu_id = smp_processor_id(); + q_index = (u32) (cpu_id) % (pm8001_ha->max_q_num); circularQ = &pm8001_ha->inbnd_q_tbl[q_index]; /* Check if encryption is set */ @@ -4576,9 +4588,10 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, ssp_cmd.esgl = 0; } } - q_index = (u32) (pm8001_dev->id & 0x00ffffff) % PM8001_MAX_OUTB_NUM; + spin_lock_irqsave(&circularQ->iq_lock, flags); ret = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &ssp_cmd, sizeof(ssp_cmd), q_index); + spin_unlock_irqrestore(&circularQ->iq_lock, flags); return ret; } @@ -4590,7 +4603,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, struct pm8001_device *pm8001_ha_dev = dev->lldd_dev; u32 tag = ccb->ccb_tag; int ret; - u32 q_index; + u32 q_index, cpu_id; struct sata_start_req sata_cmd; u32 hdr_tag, ncg_tag = 0; u64 phys_addr, start_addr, end_addr; @@ -4601,7 +4614,8 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, unsigned long flags; u32 opc = OPC_INB_SATA_HOST_OPSTART; memset(&sata_cmd, 0, sizeof(sata_cmd)); - q_index = (u32) (pm8001_ha_dev->id & 0x00ffffff) % PM8001_MAX_INB_NUM; + cpu_id = smp_processor_id(); + q_index = (u32) (cpu_id) % (pm8001_ha->max_q_num); circularQ = &pm8001_ha->inbnd_q_tbl[q_index]; if (task->data_dir == DMA_NONE) { @@ -4817,9 +4831,10 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, } } } - q_index = (u32) (pm8001_ha_dev->id & 0x00ffffff) % PM8001_MAX_OUTB_NUM; + spin_lock_irqsave(&circularQ->iq_lock, flags); ret = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &sata_cmd, sizeof(sata_cmd), q_index); + spin_unlock_irqrestore(&circularQ->iq_lock, flags); return ret; } -- 2.16.3 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH 1/4] pm80xx : Increase number of supported queues. 2020-09-25 6:16 ` [PATCH 1/4] pm80xx : Increase number of supported queues Viswas G @ 2020-09-25 9:14 ` John Garry 2020-10-05 11:13 ` Jinpu Wang 1 sibling, 0 replies; 16+ messages in thread From: John Garry @ 2020-09-25 9:14 UTC (permalink / raw) To: Viswas G, linux-scsi Cc: Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang On 25/09/2020 07:16, Viswas G wrote: > From: Viswas G<Viswas.G@microchip.com> > > The number of supported Inbound and outbound queues is increased to 64, > and the number of queues used are decided based on number of CPU cores > online and number of msix allocated. Added locks per queue instead of > global lock. > > Signed-off-by: Viswas G<Viswas.G@microchip.com> > Signed-off-by: Ruksar Devadi<Ruksar.devadi@microchip.com> In this series https://lore.kernel.org/linux-scsi/655385f6-ac59-da3b-11f9-1f8382e5c35b@huawei.com/T/#m174b66e43cb437b3fd0845742be6b011a69abb11, we allow the LLDD to expose hw queues to the upper layer for when the host requires a unique hostwide tag (which pm80xx seems to). This has certain advantages. Have you considered making a similar change here? Thanks, John ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 1/4] pm80xx : Increase number of supported queues. 2020-09-25 6:16 ` [PATCH 1/4] pm80xx : Increase number of supported queues Viswas G 2020-09-25 9:14 ` John Garry @ 2020-10-05 11:13 ` Jinpu Wang 1 sibling, 0 replies; 16+ messages in thread From: Jinpu Wang @ 2020-10-05 11:13 UTC (permalink / raw) To: Viswas G Cc: Linux SCSI Mailinglist, Vasanthalakshmi.Tharmarajan, Viswas G, Ruksar.devadi On Fri, Sep 25, 2020 at 8:06 AM Viswas G <Viswas.G@microchip.com.com> wrote: > > From: Viswas G <Viswas.G@microchip.com> > > The number of supported Inbound and outbound queues is increased to 64, > and the number of queues used are decided based on number of CPU cores > online and number of msix allocated. Added locks per queue instead of > global lock. > Please improve the commit message to explain why you did the change, not only what has been changed. > Signed-off-by: Viswas G <Viswas.G@microchip.com> > Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> The patch itself looks fine to me. Thank you > --- > drivers/scsi/pm8001/pm8001_ctl.c | 6 +- > drivers/scsi/pm8001/pm8001_defs.h | 17 +++--- > drivers/scsi/pm8001/pm8001_hwi.c | 32 ++++++----- > drivers/scsi/pm8001/pm8001_init.c | 117 ++++++++++++++++++++++++-------------- > drivers/scsi/pm8001/pm8001_sas.h | 11 +++- > drivers/scsi/pm8001/pm80xx_hwi.c | 81 +++++++++++++++----------- > 6 files changed, 160 insertions(+), 104 deletions(-) > > diff --git a/drivers/scsi/pm8001/pm8001_ctl.c b/drivers/scsi/pm8001/pm8001_ctl.c > index 77c805db2724..3587f7c8a428 100644 > --- a/drivers/scsi/pm8001/pm8001_ctl.c > +++ b/drivers/scsi/pm8001/pm8001_ctl.c > @@ -408,9 +408,10 @@ static ssize_t pm8001_ctl_ib_queue_log_show(struct device *cdev, > int offset; > char *str = buf; > int start = 0; > + u32 ib_offset = pm8001_ha->ib_offset; > #define IB_MEMMAP(c) \ > (*(u32 *)((u8 *)pm8001_ha-> \ > - memoryMap.region[IB].virt_ptr + \ > + memoryMap.region[ib_offset].virt_ptr + \ > pm8001_ha->evtlog_ib_offset + (c))) > > for (offset = 0; offset < IB_OB_READ_TIMES; offset++) { > @@ -442,9 +443,10 @@ static ssize_t pm8001_ctl_ob_queue_log_show(struct device *cdev, > int offset; > char *str = buf; > int start = 0; > + u32 ob_offset = pm8001_ha->ob_offset; > #define OB_MEMMAP(c) \ > (*(u32 *)((u8 *)pm8001_ha-> \ > - memoryMap.region[OB].virt_ptr + \ > + memoryMap.region[ob_offset].virt_ptr + \ > pm8001_ha->evtlog_ob_offset + (c))) > > for (offset = 0; offset < IB_OB_READ_TIMES; offset++) { > diff --git a/drivers/scsi/pm8001/pm8001_defs.h b/drivers/scsi/pm8001/pm8001_defs.h > index 1c7f15fd69ce..a4f52a5a449e 100644 > --- a/drivers/scsi/pm8001/pm8001_defs.h > +++ b/drivers/scsi/pm8001/pm8001_defs.h > @@ -77,10 +77,8 @@ enum port_type { > /* driver compile-time configuration */ > #define PM8001_MAX_CCB 256 /* max ccbs supported */ > #define PM8001_MPI_QUEUE 1024 /* maximum mpi queue entries */ > -#define PM8001_MAX_INB_NUM 1 > -#define PM8001_MAX_OUTB_NUM 1 > -#define PM8001_MAX_SPCV_INB_NUM 1 > -#define PM8001_MAX_SPCV_OUTB_NUM 4 > +#define PM8001_MAX_INB_NUM 64 > +#define PM8001_MAX_OUTB_NUM 64 > #define PM8001_CAN_QUEUE 508 /* SCSI Queue depth */ > > /* Inbound/Outbound queue size */ > @@ -94,11 +92,6 @@ enum port_type { > #define PM8001_MAX_MSIX_VEC 64 /* max msi-x int for spcv/ve */ > > #define USI_MAX_MEMCNT_BASE 5 > -#define IB (USI_MAX_MEMCNT_BASE + 1) > -#define CI (IB + PM8001_MAX_SPCV_INB_NUM) > -#define OB (CI + PM8001_MAX_SPCV_INB_NUM) > -#define PI (OB + PM8001_MAX_SPCV_OUTB_NUM) > -#define USI_MAX_MEMCNT (PI + PM8001_MAX_SPCV_OUTB_NUM) > #define CONFIG_SCSI_PM8001_MAX_DMA_SG 528 > #define PM8001_MAX_DMA_SG CONFIG_SCSI_PM8001_MAX_DMA_SG > enum memory_region_num { > @@ -112,6 +105,12 @@ enum memory_region_num { > }; > #define PM8001_EVENT_LOG_SIZE (128 * 1024) > > +/** > + * maximum DMA memory regions(number of IBQ + number of IBQ CI > + * + number of OBQ + number of OBQ PI) > + */ > +#define USI_MAX_MEMCNT (USI_MAX_MEMCNT_BASE + 1 + ((2 * PM8001_MAX_INB_NUM) \ > + + (2 * PM8001_MAX_OUTB_NUM))) > /*error code*/ > enum mpi_err { > MPI_IO_STATUS_SUCCESS = 0x0, > diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c > index e9a939230b15..e9106575a30f 100644 > --- a/drivers/scsi/pm8001/pm8001_hwi.c > +++ b/drivers/scsi/pm8001/pm8001_hwi.c > @@ -189,6 +189,10 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) > u32 offsetib, offsetob; > void __iomem *addressib = pm8001_ha->inbnd_q_tbl_addr; > void __iomem *addressob = pm8001_ha->outbnd_q_tbl_addr; > + u32 ib_offset = pm8001_ha->ib_offset; > + u32 ob_offset = pm8001_ha->ob_offset; > + u32 ci_offset = pm8001_ha->ci_offset; > + u32 pi_offset = pm8001_ha->pi_offset; > > pm8001_ha->main_cfg_tbl.pm8001_tbl.inbound_q_nppd_hppd = 0; > pm8001_ha->main_cfg_tbl.pm8001_tbl.outbound_hw_event_pid0_3 = 0; > @@ -223,19 +227,19 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) > pm8001_ha->inbnd_q_tbl[i].element_pri_size_cnt = > PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x00<<30); > pm8001_ha->inbnd_q_tbl[i].upper_base_addr = > - pm8001_ha->memoryMap.region[IB + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_hi; > pm8001_ha->inbnd_q_tbl[i].lower_base_addr = > - pm8001_ha->memoryMap.region[IB + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_lo; > pm8001_ha->inbnd_q_tbl[i].base_virt = > - (u8 *)pm8001_ha->memoryMap.region[IB + i].virt_ptr; > + (u8 *)pm8001_ha->memoryMap.region[ib_offset + i].virt_ptr; > pm8001_ha->inbnd_q_tbl[i].total_length = > - pm8001_ha->memoryMap.region[IB + i].total_len; > + pm8001_ha->memoryMap.region[ib_offset + i].total_len; > pm8001_ha->inbnd_q_tbl[i].ci_upper_base_addr = > - pm8001_ha->memoryMap.region[CI + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_hi; > pm8001_ha->inbnd_q_tbl[i].ci_lower_base_addr = > - pm8001_ha->memoryMap.region[CI + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_lo; > pm8001_ha->inbnd_q_tbl[i].ci_virt = > - pm8001_ha->memoryMap.region[CI + i].virt_ptr; > + pm8001_ha->memoryMap.region[ci_offset + i].virt_ptr; > offsetib = i * 0x20; > pm8001_ha->inbnd_q_tbl[i].pi_pci_bar = > get_pci_bar_index(pm8001_mr32(addressib, > @@ -249,21 +253,21 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) > pm8001_ha->outbnd_q_tbl[i].element_size_cnt = > PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x01<<30); > pm8001_ha->outbnd_q_tbl[i].upper_base_addr = > - pm8001_ha->memoryMap.region[OB + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_hi; > pm8001_ha->outbnd_q_tbl[i].lower_base_addr = > - pm8001_ha->memoryMap.region[OB + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_lo; > pm8001_ha->outbnd_q_tbl[i].base_virt = > - (u8 *)pm8001_ha->memoryMap.region[OB + i].virt_ptr; > + (u8 *)pm8001_ha->memoryMap.region[ob_offset + i].virt_ptr; > pm8001_ha->outbnd_q_tbl[i].total_length = > - pm8001_ha->memoryMap.region[OB + i].total_len; > + pm8001_ha->memoryMap.region[ob_offset + i].total_len; > pm8001_ha->outbnd_q_tbl[i].pi_upper_base_addr = > - pm8001_ha->memoryMap.region[PI + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_hi; > pm8001_ha->outbnd_q_tbl[i].pi_lower_base_addr = > - pm8001_ha->memoryMap.region[PI + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_lo; > pm8001_ha->outbnd_q_tbl[i].interrup_vec_cnt_delay = > 0 | (10 << 16) | (i << 24); > pm8001_ha->outbnd_q_tbl[i].pi_virt = > - pm8001_ha->memoryMap.region[PI + i].virt_ptr; > + pm8001_ha->memoryMap.region[pi_offset + i].virt_ptr; > offsetob = i * 0x24; > pm8001_ha->outbnd_q_tbl[i].ci_pci_bar = > get_pci_bar_index(pm8001_mr32(addressob, > diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c > index 20fa96cbc9d3..c744f846e08d 100644 > --- a/drivers/scsi/pm8001/pm8001_init.c > +++ b/drivers/scsi/pm8001/pm8001_init.c > @@ -264,12 +264,36 @@ static u32 pm8001_request_irq(struct pm8001_hba_info *pm8001_ha); > static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > const struct pci_device_id *ent) > { > - int i; > + int i, count = 0, rc = 0; > + u32 ci_offset, ib_offset, ob_offset, pi_offset; > + struct inbound_queue_table *circularQ; > + > spin_lock_init(&pm8001_ha->lock); > spin_lock_init(&pm8001_ha->bitmap_lock); > PM8001_INIT_DBG(pm8001_ha, > pm8001_printk("pm8001_alloc: PHY:%x\n", > pm8001_ha->chip->n_phy)); > + > + /* Setup Interrupt */ > + rc = pm8001_setup_irq(pm8001_ha); > + if (rc) { > + PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( > + "pm8001_setup_irq failed [ret: %d]\n", rc)); > + goto err_out_shost; > + } > + /* Request Interrupt */ > + rc = pm8001_request_irq(pm8001_ha); > + if (rc) > + goto err_out_shost; > + > + count = pm8001_ha->max_q_num; > + /* Queues are chosen based on the number of cores/msix availability */ > + ib_offset = pm8001_ha->ib_offset = USI_MAX_MEMCNT_BASE + 1; > + ci_offset = pm8001_ha->ci_offset = ib_offset + count; > + ob_offset = pm8001_ha->ob_offset = ci_offset + count; > + pi_offset = pm8001_ha->pi_offset = ob_offset + count; > + pm8001_ha->max_memcnt = pi_offset + count; > + > for (i = 0; i < pm8001_ha->chip->n_phy; i++) { > pm8001_phy_init(pm8001_ha, i); > pm8001_ha->port[i].wide_port_phymap = 0; > @@ -293,54 +317,62 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > pm8001_ha->memoryMap.region[IOP].total_len = PM8001_EVENT_LOG_SIZE; > pm8001_ha->memoryMap.region[IOP].alignment = 32; > > - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) { > + for (i = 0; i < count; i++) { > + circularQ = &pm8001_ha->inbnd_q_tbl[i]; > + spin_lock_init(&circularQ->iq_lock); > /* MPI Memory region 3 for consumer Index of inbound queues */ > - pm8001_ha->memoryMap.region[CI+i].num_elements = 1; > - pm8001_ha->memoryMap.region[CI+i].element_size = 4; > - pm8001_ha->memoryMap.region[CI+i].total_len = 4; > - pm8001_ha->memoryMap.region[CI+i].alignment = 4; > + pm8001_ha->memoryMap.region[ci_offset+i].num_elements = 1; > + pm8001_ha->memoryMap.region[ci_offset+i].element_size = 4; > + pm8001_ha->memoryMap.region[ci_offset+i].total_len = 4; > + pm8001_ha->memoryMap.region[ci_offset+i].alignment = 4; > > if ((ent->driver_data) != chip_8001) { > /* MPI Memory region 5 inbound queues */ > - pm8001_ha->memoryMap.region[IB+i].num_elements = > + pm8001_ha->memoryMap.region[ib_offset+i].num_elements = > PM8001_MPI_QUEUE; > - pm8001_ha->memoryMap.region[IB+i].element_size = 128; > - pm8001_ha->memoryMap.region[IB+i].total_len = > + pm8001_ha->memoryMap.region[ib_offset+i].element_size > + = 128; > + pm8001_ha->memoryMap.region[ib_offset+i].total_len = > PM8001_MPI_QUEUE * 128; > - pm8001_ha->memoryMap.region[IB+i].alignment = 128; > + pm8001_ha->memoryMap.region[ib_offset+i].alignment > + = 128; > } else { > - pm8001_ha->memoryMap.region[IB+i].num_elements = > + pm8001_ha->memoryMap.region[ib_offset+i].num_elements = > PM8001_MPI_QUEUE; > - pm8001_ha->memoryMap.region[IB+i].element_size = 64; > - pm8001_ha->memoryMap.region[IB+i].total_len = > + pm8001_ha->memoryMap.region[ib_offset+i].element_size > + = 64; > + pm8001_ha->memoryMap.region[ib_offset+i].total_len = > PM8001_MPI_QUEUE * 64; > - pm8001_ha->memoryMap.region[IB+i].alignment = 64; > + pm8001_ha->memoryMap.region[ib_offset+i].alignment = 64; > } > } > > - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) { > + for (i = 0; i < count; i++) { > /* MPI Memory region 4 for producer Index of outbound queues */ > - pm8001_ha->memoryMap.region[PI+i].num_elements = 1; > - pm8001_ha->memoryMap.region[PI+i].element_size = 4; > - pm8001_ha->memoryMap.region[PI+i].total_len = 4; > - pm8001_ha->memoryMap.region[PI+i].alignment = 4; > + pm8001_ha->memoryMap.region[pi_offset+i].num_elements = 1; > + pm8001_ha->memoryMap.region[pi_offset+i].element_size = 4; > + pm8001_ha->memoryMap.region[pi_offset+i].total_len = 4; > + pm8001_ha->memoryMap.region[pi_offset+i].alignment = 4; > > if (ent->driver_data != chip_8001) { > /* MPI Memory region 6 Outbound queues */ > - pm8001_ha->memoryMap.region[OB+i].num_elements = > + pm8001_ha->memoryMap.region[ob_offset+i].num_elements = > PM8001_MPI_QUEUE; > - pm8001_ha->memoryMap.region[OB+i].element_size = 128; > - pm8001_ha->memoryMap.region[OB+i].total_len = > + pm8001_ha->memoryMap.region[ob_offset+i].element_size > + = 128; > + pm8001_ha->memoryMap.region[ob_offset+i].total_len = > PM8001_MPI_QUEUE * 128; > - pm8001_ha->memoryMap.region[OB+i].alignment = 128; > + pm8001_ha->memoryMap.region[ob_offset+i].alignment > + = 128; > } else { > /* MPI Memory region 6 Outbound queues */ > - pm8001_ha->memoryMap.region[OB+i].num_elements = > + pm8001_ha->memoryMap.region[ob_offset+i].num_elements = > PM8001_MPI_QUEUE; > - pm8001_ha->memoryMap.region[OB+i].element_size = 64; > - pm8001_ha->memoryMap.region[OB+i].total_len = > + pm8001_ha->memoryMap.region[ob_offset+i].element_size > + = 64; > + pm8001_ha->memoryMap.region[ob_offset+i].total_len = > PM8001_MPI_QUEUE * 64; > - pm8001_ha->memoryMap.region[OB+i].alignment = 64; > + pm8001_ha->memoryMap.region[ob_offset+i].alignment = 64; > } > > } > @@ -369,7 +401,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > pm8001_ha->memoryMap.region[FORENSIC_MEM].total_len = 0x10000; > pm8001_ha->memoryMap.region[FORENSIC_MEM].element_size = 0x10000; > pm8001_ha->memoryMap.region[FORENSIC_MEM].alignment = 0x10000; > - for (i = 0; i < USI_MAX_MEMCNT; i++) { > + for (i = 0; i < pm8001_ha->max_memcnt; i++) { > if (pm8001_mem_alloc(pm8001_ha->pdev, > &pm8001_ha->memoryMap.region[i].virt_ptr, > &pm8001_ha->memoryMap.region[i].phys_addr, > @@ -405,6 +437,8 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > /* Initialize tags */ > pm8001_tag_init(pm8001_ha); > return 0; > +err_out_shost: > + scsi_remove_host(pm8001_ha->shost); > err_out: > return 1; > } > @@ -899,7 +933,8 @@ static int pm8001_configure_phy_settings(struct pm8001_hba_info *pm8001_ha) > static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) > { > u32 number_of_intr; > - int rc; > + int rc, cpu_online_count; > + unsigned int allocated_irq_vectors; > > /* SPCv controllers supports 64 msi-x */ > if (pm8001_ha->chip_id == chip_8001) { > @@ -908,13 +943,21 @@ static u32 pm8001_setup_msix(struct pm8001_hba_info *pm8001_ha) > number_of_intr = PM8001_MAX_MSIX_VEC; > } > > + cpu_online_count = num_online_cpus(); > + number_of_intr = min_t(int, cpu_online_count, number_of_intr); > rc = pci_alloc_irq_vectors(pm8001_ha->pdev, number_of_intr, > number_of_intr, PCI_IRQ_MSIX); > - number_of_intr = rc; > + allocated_irq_vectors = rc; > if (rc < 0) > return rc; > + > + /*Assigns the number of interrupts */ > + number_of_intr = min_t(int, allocated_irq_vectors, number_of_intr); > pm8001_ha->number_of_intr = number_of_intr; > > + /*maximum queue number updating in HBA structure */ > + pm8001_ha->max_q_num = number_of_intr; > + > PM8001_INIT_DBG(pm8001_ha, pm8001_printk( > "pci_alloc_irq_vectors request ret:%d no of intr %d\n", > rc, pm8001_ha->number_of_intr)); > @@ -1069,13 +1112,6 @@ static int pm8001_pci_probe(struct pci_dev *pdev, > rc = -ENOMEM; > goto err_out_free; > } > - /* Setup Interrupt */ > - rc = pm8001_setup_irq(pm8001_ha); > - if (rc) { > - PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( > - "pm8001_setup_irq failed [ret: %d]\n", rc)); > - goto err_out_shost; > - } > > PM8001_CHIP_DISP->chip_soft_rst(pm8001_ha); > rc = PM8001_CHIP_DISP->chip_init(pm8001_ha); > @@ -1088,13 +1124,6 @@ static int pm8001_pci_probe(struct pci_dev *pdev, > rc = scsi_add_host(shost, &pdev->dev); > if (rc) > goto err_out_ha_free; > - /* Request Interrupt */ > - rc = pm8001_request_irq(pm8001_ha); > - if (rc) { > - PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( > - "pm8001_request_irq failed [ret: %d]\n", rc)); > - goto err_out_shost; > - } > > PM8001_CHIP_DISP->interrupt_enable(pm8001_ha, 0); > if (pm8001_ha->chip_id != chip_8001) { > diff --git a/drivers/scsi/pm8001/pm8001_sas.h b/drivers/scsi/pm8001/pm8001_sas.h > index ae7ba9b3c4bc..bdfce3c3f619 100644 > --- a/drivers/scsi/pm8001/pm8001_sas.h > +++ b/drivers/scsi/pm8001/pm8001_sas.h > @@ -468,6 +468,7 @@ struct inbound_queue_table { > u32 reserved; > __le32 consumer_index; > u32 producer_idx; > + spinlock_t iq_lock; > }; > struct outbound_queue_table { > u32 element_size_cnt; > @@ -524,8 +525,8 @@ struct pm8001_hba_info { > void __iomem *fatal_tbl_addr; /*MPI IVT Table Addr */ > union main_cfg_table main_cfg_tbl; > union general_status_table gs_tbl; > - struct inbound_queue_table inbnd_q_tbl[PM8001_MAX_SPCV_INB_NUM]; > - struct outbound_queue_table outbnd_q_tbl[PM8001_MAX_SPCV_OUTB_NUM]; > + struct inbound_queue_table inbnd_q_tbl[PM8001_MAX_INB_NUM]; > + struct outbound_queue_table outbnd_q_tbl[PM8001_MAX_OUTB_NUM]; > struct sas_phy_attribute_table phy_attr_table; > /* MPI SAS PHY attributes */ > u8 sas_addr[SAS_ADDR_SIZE]; > @@ -561,6 +562,12 @@ struct pm8001_hba_info { > u32 reset_in_progress; > u32 non_fatal_count; > u32 non_fatal_read_length; > + u32 max_q_num; > + u32 ib_offset; > + u32 ob_offset; > + u32 ci_offset; > + u32 pi_offset; > + u32 max_memcnt; > }; > > struct pm8001_work { > diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c > index b42f41d1ed49..26e9e8877107 100644 > --- a/drivers/scsi/pm8001/pm80xx_hwi.c > +++ b/drivers/scsi/pm8001/pm80xx_hwi.c > @@ -720,7 +720,7 @@ static void read_inbnd_queue_table(struct pm8001_hba_info *pm8001_ha) > { > int i; > void __iomem *address = pm8001_ha->inbnd_q_tbl_addr; > - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) { > + for (i = 0; i < PM8001_MAX_INB_NUM; i++) { > u32 offset = i * 0x20; > pm8001_ha->inbnd_q_tbl[i].pi_pci_bar = > get_pci_bar_index(pm8001_mr32(address, > @@ -738,7 +738,7 @@ static void read_outbnd_queue_table(struct pm8001_hba_info *pm8001_ha) > { > int i; > void __iomem *address = pm8001_ha->outbnd_q_tbl_addr; > - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) { > + for (i = 0; i < PM8001_MAX_OUTB_NUM; i++) { > u32 offset = i * 0x24; > pm8001_ha->outbnd_q_tbl[i].ci_pci_bar = > get_pci_bar_index(pm8001_mr32(address, > @@ -758,6 +758,10 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) > u32 offsetib, offsetob; > void __iomem *addressib = pm8001_ha->inbnd_q_tbl_addr; > void __iomem *addressob = pm8001_ha->outbnd_q_tbl_addr; > + u32 ib_offset = pm8001_ha->ib_offset; > + u32 ob_offset = pm8001_ha->ob_offset; > + u32 ci_offset = pm8001_ha->ci_offset; > + u32 pi_offset = pm8001_ha->pi_offset; > > pm8001_ha->main_cfg_tbl.pm80xx_tbl.upper_event_log_addr = > pm8001_ha->memoryMap.region[AAP1].phys_addr_hi; > @@ -778,23 +782,23 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) > /* Disable end to end CRC checking */ > pm8001_ha->main_cfg_tbl.pm80xx_tbl.crc_core_dump = (0x1 << 16); > > - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) { > + for (i = 0; i < pm8001_ha->max_q_num; i++) { > pm8001_ha->inbnd_q_tbl[i].element_pri_size_cnt = > PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x00<<30); > pm8001_ha->inbnd_q_tbl[i].upper_base_addr = > - pm8001_ha->memoryMap.region[IB + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_hi; > pm8001_ha->inbnd_q_tbl[i].lower_base_addr = > - pm8001_ha->memoryMap.region[IB + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[ib_offset + i].phys_addr_lo; > pm8001_ha->inbnd_q_tbl[i].base_virt = > - (u8 *)pm8001_ha->memoryMap.region[IB + i].virt_ptr; > + (u8 *)pm8001_ha->memoryMap.region[ib_offset + i].virt_ptr; > pm8001_ha->inbnd_q_tbl[i].total_length = > - pm8001_ha->memoryMap.region[IB + i].total_len; > + pm8001_ha->memoryMap.region[ib_offset + i].total_len; > pm8001_ha->inbnd_q_tbl[i].ci_upper_base_addr = > - pm8001_ha->memoryMap.region[CI + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_hi; > pm8001_ha->inbnd_q_tbl[i].ci_lower_base_addr = > - pm8001_ha->memoryMap.region[CI + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[ci_offset + i].phys_addr_lo; > pm8001_ha->inbnd_q_tbl[i].ci_virt = > - pm8001_ha->memoryMap.region[CI + i].virt_ptr; > + pm8001_ha->memoryMap.region[ci_offset + i].virt_ptr; > offsetib = i * 0x20; > pm8001_ha->inbnd_q_tbl[i].pi_pci_bar = > get_pci_bar_index(pm8001_mr32(addressib, > @@ -809,25 +813,25 @@ static void init_default_table_values(struct pm8001_hba_info *pm8001_ha) > pm8001_ha->inbnd_q_tbl[i].pi_pci_bar, > pm8001_ha->inbnd_q_tbl[i].pi_offset)); > } > - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) { > + for (i = 0; i < pm8001_ha->max_q_num; i++) { > pm8001_ha->outbnd_q_tbl[i].element_size_cnt = > PM8001_MPI_QUEUE | (pm8001_ha->iomb_size << 16) | (0x01<<30); > pm8001_ha->outbnd_q_tbl[i].upper_base_addr = > - pm8001_ha->memoryMap.region[OB + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_hi; > pm8001_ha->outbnd_q_tbl[i].lower_base_addr = > - pm8001_ha->memoryMap.region[OB + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[ob_offset + i].phys_addr_lo; > pm8001_ha->outbnd_q_tbl[i].base_virt = > - (u8 *)pm8001_ha->memoryMap.region[OB + i].virt_ptr; > + (u8 *)pm8001_ha->memoryMap.region[ob_offset + i].virt_ptr; > pm8001_ha->outbnd_q_tbl[i].total_length = > - pm8001_ha->memoryMap.region[OB + i].total_len; > + pm8001_ha->memoryMap.region[ob_offset + i].total_len; > pm8001_ha->outbnd_q_tbl[i].pi_upper_base_addr = > - pm8001_ha->memoryMap.region[PI + i].phys_addr_hi; > + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_hi; > pm8001_ha->outbnd_q_tbl[i].pi_lower_base_addr = > - pm8001_ha->memoryMap.region[PI + i].phys_addr_lo; > + pm8001_ha->memoryMap.region[pi_offset + i].phys_addr_lo; > /* interrupt vector based on oq */ > pm8001_ha->outbnd_q_tbl[i].interrup_vec_cnt_delay = (i << 24); > pm8001_ha->outbnd_q_tbl[i].pi_virt = > - pm8001_ha->memoryMap.region[PI + i].virt_ptr; > + pm8001_ha->memoryMap.region[pi_offset + i].virt_ptr; > offsetob = i * 0x24; > pm8001_ha->outbnd_q_tbl[i].ci_pci_bar = > get_pci_bar_index(pm8001_mr32(addressob, > @@ -871,7 +875,7 @@ static void update_main_config_table(struct pm8001_hba_info *pm8001_ha) > pm8001_ha->main_cfg_tbl.pm80xx_tbl.pcs_event_log_severity); > /* Update Fatal error interrupt vector */ > pm8001_ha->main_cfg_tbl.pm80xx_tbl.fatal_err_interrupt |= > - ((pm8001_ha->number_of_intr - 1) << 8); > + ((pm8001_ha->max_q_num - 1) << 8); > pm8001_mw32(address, MAIN_FATAL_ERROR_INTERRUPT, > pm8001_ha->main_cfg_tbl.pm80xx_tbl.fatal_err_interrupt); > PM8001_DEV_DBG(pm8001_ha, pm8001_printk( > @@ -1010,8 +1014,12 @@ static int mpi_init_check(struct pm8001_hba_info *pm8001_ha) > value &= SPCv_MSGU_CFG_TABLE_UPDATE; > } while ((value != 0) && (--max_wait_count)); > > - if (!max_wait_count) > - return -1; > + if (!max_wait_count) { > + /* additional check */ > + PM8001_FAIL_DBG(pm8001_ha, pm8001_printk( > + "Inb doorbell clear not toggled[value:%x]\n", value)); > + return -EBUSY; > + } > /* check the MPI-State for initialization upto 100ms*/ > max_wait_count = 100 * 1000;/* 100 msec */ > do { > @@ -1022,12 +1030,12 @@ static int mpi_init_check(struct pm8001_hba_info *pm8001_ha) > } while ((GST_MPI_STATE_INIT != > (gst_len_mpistate & GST_MPI_STATE_MASK)) && (--max_wait_count)); > if (!max_wait_count) > - return -1; > + return -EBUSY; > > /* check MPI Initialization error */ > gst_len_mpistate = gst_len_mpistate >> 16; > if (0x0000 != gst_len_mpistate) > - return -1; > + return -EBUSY; > > return 0; > } > @@ -1469,11 +1477,10 @@ static int pm80xx_chip_init(struct pm8001_hba_info *pm8001_ha) > > /* update main config table ,inbound table and outbound table */ > update_main_config_table(pm8001_ha); > - for (i = 0; i < PM8001_MAX_SPCV_INB_NUM; i++) > + for (i = 0; i < pm8001_ha->max_q_num; i++) { > update_inbnd_queue_table(pm8001_ha, i); > - for (i = 0; i < PM8001_MAX_SPCV_OUTB_NUM; i++) > update_outbnd_queue_table(pm8001_ha, i); > - > + } > /* notify firmware update finished and check initialization status */ > if (0 == mpi_init_check(pm8001_ha)) { > PM8001_INIT_DBG(pm8001_ha, > @@ -4191,7 +4198,7 @@ static int process_oq(struct pm8001_hba_info *pm8001_ha, u8 vec) > unsigned long flags; > u32 regval; > > - if (vec == (pm8001_ha->number_of_intr - 1)) { > + if (vec == (pm8001_ha->max_q_num - 1)) { > regval = pm8001_cr32(pm8001_ha, 0, MSGU_SCRATCH_PAD_1); > if ((regval & SCRATCH_PAD_MIPSALL_READY) != > SCRATCH_PAD_MIPSALL_READY) { > @@ -4274,6 +4281,7 @@ static int pm80xx_chip_smp_req(struct pm8001_hba_info *pm8001_ha, > char *preq_dma_addr = NULL; > __le64 tmp_addr; > u32 i, length; > + unsigned long flags; > > memset(&smp_cmd, 0, sizeof(smp_cmd)); > /* > @@ -4369,8 +4377,10 @@ static int pm80xx_chip_smp_req(struct pm8001_hba_info *pm8001_ha, > > build_smp_cmd(pm8001_dev->device_id, smp_cmd.tag, > &smp_cmd, pm8001_ha->smp_exp_mode, length); > + spin_lock_irqsave(&circularQ->iq_lock, flags); > rc = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, &smp_cmd, > sizeof(smp_cmd), 0); > + spin_unlock_irqrestore(&circularQ->iq_lock, flags); > if (rc) > goto err_out_2; > return 0; > @@ -4434,7 +4444,8 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > u64 phys_addr, start_addr, end_addr; > u32 end_addr_high, end_addr_low; > struct inbound_queue_table *circularQ; > - u32 q_index; > + unsigned long flags; > + u32 q_index, cpu_id; > u32 opc = OPC_INB_SSPINIIOSTART; > memset(&ssp_cmd, 0, sizeof(ssp_cmd)); > memcpy(ssp_cmd.ssp_iu.lun, task->ssp_task.LUN, 8); > @@ -4453,7 +4464,8 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > ssp_cmd.ssp_iu.efb_prio_attr |= (task->ssp_task.task_attr & 7); > memcpy(ssp_cmd.ssp_iu.cdb, task->ssp_task.cmd->cmnd, > task->ssp_task.cmd->cmd_len); > - q_index = (u32) (pm8001_dev->id & 0x00ffffff) % PM8001_MAX_INB_NUM; > + cpu_id = smp_processor_id(); > + q_index = (u32) (cpu_id) % (pm8001_ha->max_q_num); > circularQ = &pm8001_ha->inbnd_q_tbl[q_index]; > > /* Check if encryption is set */ > @@ -4576,9 +4588,10 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > ssp_cmd.esgl = 0; > } > } > - q_index = (u32) (pm8001_dev->id & 0x00ffffff) % PM8001_MAX_OUTB_NUM; > + spin_lock_irqsave(&circularQ->iq_lock, flags); > ret = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, > &ssp_cmd, sizeof(ssp_cmd), q_index); > + spin_unlock_irqrestore(&circularQ->iq_lock, flags); > return ret; > } > > @@ -4590,7 +4603,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > struct pm8001_device *pm8001_ha_dev = dev->lldd_dev; > u32 tag = ccb->ccb_tag; > int ret; > - u32 q_index; > + u32 q_index, cpu_id; > struct sata_start_req sata_cmd; > u32 hdr_tag, ncg_tag = 0; > u64 phys_addr, start_addr, end_addr; > @@ -4601,7 +4614,8 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > unsigned long flags; > u32 opc = OPC_INB_SATA_HOST_OPSTART; > memset(&sata_cmd, 0, sizeof(sata_cmd)); > - q_index = (u32) (pm8001_ha_dev->id & 0x00ffffff) % PM8001_MAX_INB_NUM; > + cpu_id = smp_processor_id(); > + q_index = (u32) (cpu_id) % (pm8001_ha->max_q_num); > circularQ = &pm8001_ha->inbnd_q_tbl[q_index]; > > if (task->data_dir == DMA_NONE) { > @@ -4817,9 +4831,10 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > } > } > } > - q_index = (u32) (pm8001_ha_dev->id & 0x00ffffff) % PM8001_MAX_OUTB_NUM; > + spin_lock_irqsave(&circularQ->iq_lock, flags); > ret = pm8001_mpi_build_cmd(pm8001_ha, circularQ, opc, > &sata_cmd, sizeof(sata_cmd), q_index); > + spin_unlock_irqrestore(&circularQ->iq_lock, flags); > return ret; > } > > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH 2/4] pm80xx : Remove DMA memory allocation for ccb and device 2020-09-25 6:16 [PATCH 0/4] pm80xx updates Viswas G 2020-09-25 6:13 ` Jinpu Wang 2020-09-25 6:16 ` [PATCH 1/4] pm80xx : Increase number of supported queues Viswas G @ 2020-09-25 6:16 ` Viswas G 2020-10-05 11:16 ` Jinpu Wang 2020-09-25 6:16 ` [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported Viswas G 2020-09-25 6:16 ` [PATCH 4/4] pm80xx : Driver version update Viswas G 4 siblings, 1 reply; 16+ messages in thread From: Viswas G @ 2020-09-25 6:16 UTC (permalink / raw) To: linux-scsi Cc: Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang From: Viswas G <Viswas.G@microchip.com> Removed DMA memory allocation for Devices and CCB structure, instead allocated memory outside DMA memory. Signed-off-by: Viswas G <Viswas.G@microchip.com> Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> --- drivers/scsi/pm8001/pm8001_defs.h | 8 +++---- drivers/scsi/pm8001/pm8001_init.c | 48 ++++++++++++++++++++++++--------------- 2 files changed, 33 insertions(+), 23 deletions(-) diff --git a/drivers/scsi/pm8001/pm8001_defs.h b/drivers/scsi/pm8001/pm8001_defs.h index a4f52a5a449e..1bf1bcfaf010 100644 --- a/drivers/scsi/pm8001/pm8001_defs.h +++ b/drivers/scsi/pm8001/pm8001_defs.h @@ -91,17 +91,15 @@ enum port_type { #define PM8001_MAX_DEVICES 2048 /* max supported device */ #define PM8001_MAX_MSIX_VEC 64 /* max msi-x int for spcv/ve */ -#define USI_MAX_MEMCNT_BASE 5 #define CONFIG_SCSI_PM8001_MAX_DMA_SG 528 #define PM8001_MAX_DMA_SG CONFIG_SCSI_PM8001_MAX_DMA_SG enum memory_region_num { AAP1 = 0x0, /* application acceleration processor */ IOP, /* IO processor */ NVMD, /* NVM device */ - DEV_MEM, /* memory for devices */ - CCB_MEM, /* memory for command control block */ FW_FLASH, /* memory for fw flash update */ - FORENSIC_MEM /* memory for fw forensic data */ + FORENSIC_MEM, /* memory for fw forensic data */ + USI_MAX_MEMCNT_BASE }; #define PM8001_EVENT_LOG_SIZE (128 * 1024) @@ -109,7 +107,7 @@ enum memory_region_num { * maximum DMA memory regions(number of IBQ + number of IBQ CI * + number of OBQ + number of OBQ PI) */ -#define USI_MAX_MEMCNT (USI_MAX_MEMCNT_BASE + 1 + ((2 * PM8001_MAX_INB_NUM) \ +#define USI_MAX_MEMCNT (USI_MAX_MEMCNT_BASE + ((2 * PM8001_MAX_INB_NUM) \ + (2 * PM8001_MAX_OUTB_NUM))) /*error code*/ enum mpi_err { diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c index c744f846e08d..d6789a261c1c 100644 --- a/drivers/scsi/pm8001/pm8001_init.c +++ b/drivers/scsi/pm8001/pm8001_init.c @@ -288,7 +288,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, count = pm8001_ha->max_q_num; /* Queues are chosen based on the number of cores/msix availability */ - ib_offset = pm8001_ha->ib_offset = USI_MAX_MEMCNT_BASE + 1; + ib_offset = pm8001_ha->ib_offset = USI_MAX_MEMCNT_BASE; ci_offset = pm8001_ha->ci_offset = ib_offset + count; ob_offset = pm8001_ha->ob_offset = ci_offset + count; pi_offset = pm8001_ha->pi_offset = ob_offset + count; @@ -380,19 +380,6 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, pm8001_ha->memoryMap.region[NVMD].num_elements = 1; pm8001_ha->memoryMap.region[NVMD].element_size = 4096; pm8001_ha->memoryMap.region[NVMD].total_len = 4096; - /* Memory region for devices*/ - pm8001_ha->memoryMap.region[DEV_MEM].num_elements = 1; - pm8001_ha->memoryMap.region[DEV_MEM].element_size = PM8001_MAX_DEVICES * - sizeof(struct pm8001_device); - pm8001_ha->memoryMap.region[DEV_MEM].total_len = PM8001_MAX_DEVICES * - sizeof(struct pm8001_device); - - /* Memory region for ccb_info*/ - pm8001_ha->memoryMap.region[CCB_MEM].num_elements = 1; - pm8001_ha->memoryMap.region[CCB_MEM].element_size = PM8001_MAX_CCB * - sizeof(struct pm8001_ccb_info); - pm8001_ha->memoryMap.region[CCB_MEM].total_len = PM8001_MAX_CCB * - sizeof(struct pm8001_ccb_info); /* Memory region for fw flash */ pm8001_ha->memoryMap.region[FW_FLASH].total_len = 4096; @@ -416,18 +403,30 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, } } - pm8001_ha->devices = pm8001_ha->memoryMap.region[DEV_MEM].virt_ptr; + /* Memory region for devices*/ + pm8001_ha->devices = kzalloc(PM8001_MAX_DEVICES + * sizeof(struct pm8001_device), GFP_KERNEL); + if (!pm8001_ha->devices) { + rc = -ENOMEM; + goto err_out_nodev; + } for (i = 0; i < PM8001_MAX_DEVICES; i++) { pm8001_ha->devices[i].dev_type = SAS_PHY_UNUSED; pm8001_ha->devices[i].id = i; pm8001_ha->devices[i].device_id = PM8001_MAX_DEVICES; pm8001_ha->devices[i].running_req = 0; } - pm8001_ha->ccb_info = pm8001_ha->memoryMap.region[CCB_MEM].virt_ptr; + /* Memory region for ccb_info*/ + pm8001_ha->ccb_info = kzalloc(PM8001_MAX_CCB + * sizeof(struct pm8001_ccb_info), GFP_KERNEL); + if (!pm8001_ha->ccb_info) { + rc = -ENOMEM; + goto err_out_noccb; + } for (i = 0; i < PM8001_MAX_CCB; i++) { pm8001_ha->ccb_info[i].ccb_dma_handle = - pm8001_ha->memoryMap.region[CCB_MEM].phys_addr + - i * sizeof(struct pm8001_ccb_info); + virt_to_phys(pm8001_ha->ccb_info) + + (i * sizeof(struct pm8001_ccb_info)); pm8001_ha->ccb_info[i].task = NULL; pm8001_ha->ccb_info[i].ccb_tag = 0xffffffff; pm8001_ha->ccb_info[i].device = NULL; @@ -437,8 +436,21 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, /* Initialize tags */ pm8001_tag_init(pm8001_ha); return 0; + +err_out_noccb: + kfree(pm8001_ha->devices); err_out_shost: scsi_remove_host(pm8001_ha->shost); +err_out_nodev: + for (i = 0; i < pm8001_ha->max_memcnt; i++) { + if (pm8001_ha->memoryMap.region[i].virt_ptr != NULL) { + pci_free_consistent(pm8001_ha->pdev, + (pm8001_ha->memoryMap.region[i].total_len + + pm8001_ha->memoryMap.region[i].alignment), + pm8001_ha->memoryMap.region[i].virt_ptr, + pm8001_ha->memoryMap.region[i].phys_addr); + } + } err_out: return 1; } -- 2.16.3 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH 2/4] pm80xx : Remove DMA memory allocation for ccb and device 2020-09-25 6:16 ` [PATCH 2/4] pm80xx : Remove DMA memory allocation for ccb and device Viswas G @ 2020-10-05 11:16 ` Jinpu Wang 0 siblings, 0 replies; 16+ messages in thread From: Jinpu Wang @ 2020-10-05 11:16 UTC (permalink / raw) To: Viswas G Cc: Linux SCSI Mailinglist, Vasanthalakshmi.Tharmarajan, Viswas G, Ruksar.devadi On Fri, Sep 25, 2020 at 8:06 AM Viswas G <Viswas.G@microchip.com.com> wrote: > > From: Viswas G <Viswas.G@microchip.com> > > Removed DMA memory allocation for Devices and CCB > structure, instead allocated memory outside DMA memory. > > Signed-off-by: Viswas G <Viswas.G@microchip.com> > Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> I think it's better to mention DMA memory is limited system resource, better to alloc memory outside DMA memory when possible. With the updated commit message you can add my: Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com> > --- > drivers/scsi/pm8001/pm8001_defs.h | 8 +++---- > drivers/scsi/pm8001/pm8001_init.c | 48 ++++++++++++++++++++++++--------------- > 2 files changed, 33 insertions(+), 23 deletions(-) > > diff --git a/drivers/scsi/pm8001/pm8001_defs.h b/drivers/scsi/pm8001/pm8001_defs.h > index a4f52a5a449e..1bf1bcfaf010 100644 > --- a/drivers/scsi/pm8001/pm8001_defs.h > +++ b/drivers/scsi/pm8001/pm8001_defs.h > @@ -91,17 +91,15 @@ enum port_type { > #define PM8001_MAX_DEVICES 2048 /* max supported device */ > #define PM8001_MAX_MSIX_VEC 64 /* max msi-x int for spcv/ve */ > > -#define USI_MAX_MEMCNT_BASE 5 > #define CONFIG_SCSI_PM8001_MAX_DMA_SG 528 > #define PM8001_MAX_DMA_SG CONFIG_SCSI_PM8001_MAX_DMA_SG > enum memory_region_num { > AAP1 = 0x0, /* application acceleration processor */ > IOP, /* IO processor */ > NVMD, /* NVM device */ > - DEV_MEM, /* memory for devices */ > - CCB_MEM, /* memory for command control block */ > FW_FLASH, /* memory for fw flash update */ > - FORENSIC_MEM /* memory for fw forensic data */ > + FORENSIC_MEM, /* memory for fw forensic data */ > + USI_MAX_MEMCNT_BASE > }; > #define PM8001_EVENT_LOG_SIZE (128 * 1024) > > @@ -109,7 +107,7 @@ enum memory_region_num { > * maximum DMA memory regions(number of IBQ + number of IBQ CI > * + number of OBQ + number of OBQ PI) > */ > -#define USI_MAX_MEMCNT (USI_MAX_MEMCNT_BASE + 1 + ((2 * PM8001_MAX_INB_NUM) \ > +#define USI_MAX_MEMCNT (USI_MAX_MEMCNT_BASE + ((2 * PM8001_MAX_INB_NUM) \ > + (2 * PM8001_MAX_OUTB_NUM))) > /*error code*/ > enum mpi_err { > diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c > index c744f846e08d..d6789a261c1c 100644 > --- a/drivers/scsi/pm8001/pm8001_init.c > +++ b/drivers/scsi/pm8001/pm8001_init.c > @@ -288,7 +288,7 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > > count = pm8001_ha->max_q_num; > /* Queues are chosen based on the number of cores/msix availability */ > - ib_offset = pm8001_ha->ib_offset = USI_MAX_MEMCNT_BASE + 1; > + ib_offset = pm8001_ha->ib_offset = USI_MAX_MEMCNT_BASE; > ci_offset = pm8001_ha->ci_offset = ib_offset + count; > ob_offset = pm8001_ha->ob_offset = ci_offset + count; > pi_offset = pm8001_ha->pi_offset = ob_offset + count; > @@ -380,19 +380,6 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > pm8001_ha->memoryMap.region[NVMD].num_elements = 1; > pm8001_ha->memoryMap.region[NVMD].element_size = 4096; > pm8001_ha->memoryMap.region[NVMD].total_len = 4096; > - /* Memory region for devices*/ > - pm8001_ha->memoryMap.region[DEV_MEM].num_elements = 1; > - pm8001_ha->memoryMap.region[DEV_MEM].element_size = PM8001_MAX_DEVICES * > - sizeof(struct pm8001_device); > - pm8001_ha->memoryMap.region[DEV_MEM].total_len = PM8001_MAX_DEVICES * > - sizeof(struct pm8001_device); > - > - /* Memory region for ccb_info*/ > - pm8001_ha->memoryMap.region[CCB_MEM].num_elements = 1; > - pm8001_ha->memoryMap.region[CCB_MEM].element_size = PM8001_MAX_CCB * > - sizeof(struct pm8001_ccb_info); > - pm8001_ha->memoryMap.region[CCB_MEM].total_len = PM8001_MAX_CCB * > - sizeof(struct pm8001_ccb_info); > > /* Memory region for fw flash */ > pm8001_ha->memoryMap.region[FW_FLASH].total_len = 4096; > @@ -416,18 +403,30 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > } > } > > - pm8001_ha->devices = pm8001_ha->memoryMap.region[DEV_MEM].virt_ptr; > + /* Memory region for devices*/ > + pm8001_ha->devices = kzalloc(PM8001_MAX_DEVICES > + * sizeof(struct pm8001_device), GFP_KERNEL); > + if (!pm8001_ha->devices) { > + rc = -ENOMEM; > + goto err_out_nodev; > + } > for (i = 0; i < PM8001_MAX_DEVICES; i++) { > pm8001_ha->devices[i].dev_type = SAS_PHY_UNUSED; > pm8001_ha->devices[i].id = i; > pm8001_ha->devices[i].device_id = PM8001_MAX_DEVICES; > pm8001_ha->devices[i].running_req = 0; > } > - pm8001_ha->ccb_info = pm8001_ha->memoryMap.region[CCB_MEM].virt_ptr; > + /* Memory region for ccb_info*/ > + pm8001_ha->ccb_info = kzalloc(PM8001_MAX_CCB > + * sizeof(struct pm8001_ccb_info), GFP_KERNEL); > + if (!pm8001_ha->ccb_info) { > + rc = -ENOMEM; > + goto err_out_noccb; > + } > for (i = 0; i < PM8001_MAX_CCB; i++) { > pm8001_ha->ccb_info[i].ccb_dma_handle = > - pm8001_ha->memoryMap.region[CCB_MEM].phys_addr + > - i * sizeof(struct pm8001_ccb_info); > + virt_to_phys(pm8001_ha->ccb_info) + > + (i * sizeof(struct pm8001_ccb_info)); > pm8001_ha->ccb_info[i].task = NULL; > pm8001_ha->ccb_info[i].ccb_tag = 0xffffffff; > pm8001_ha->ccb_info[i].device = NULL; > @@ -437,8 +436,21 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > /* Initialize tags */ > pm8001_tag_init(pm8001_ha); > return 0; > + > +err_out_noccb: > + kfree(pm8001_ha->devices); > err_out_shost: > scsi_remove_host(pm8001_ha->shost); > +err_out_nodev: > + for (i = 0; i < pm8001_ha->max_memcnt; i++) { > + if (pm8001_ha->memoryMap.region[i].virt_ptr != NULL) { > + pci_free_consistent(pm8001_ha->pdev, > + (pm8001_ha->memoryMap.region[i].total_len + > + pm8001_ha->memoryMap.region[i].alignment), > + pm8001_ha->memoryMap.region[i].virt_ptr, > + pm8001_ha->memoryMap.region[i].phys_addr); > + } > + } > err_out: > return 1; > } > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported 2020-09-25 6:16 [PATCH 0/4] pm80xx updates Viswas G ` (2 preceding siblings ...) 2020-09-25 6:16 ` [PATCH 2/4] pm80xx : Remove DMA memory allocation for ccb and device Viswas G @ 2020-09-25 6:16 ` Viswas G 2020-09-25 8:16 ` kernel test robot ` (2 more replies) 2020-09-25 6:16 ` [PATCH 4/4] pm80xx : Driver version update Viswas G 4 siblings, 3 replies; 16+ messages in thread From: Viswas G @ 2020-09-25 6:16 UTC (permalink / raw) To: linux-scsi Cc: Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang From: Viswas G <Viswas.G@microchip.com> Increasing the number of Outstanding IOs from 256 to 1024. CCB and tag are allocated according to outstanding IOs. Also updating the can_queue value (max_out_io - PM8001_RESERVE_SLOT) to scsi midlayer. Signed-off-by: Viswas G <Viswas.G@microchip.com> Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> --- drivers/scsi/pm8001/pm8001_defs.h | 4 +- drivers/scsi/pm8001/pm8001_hwi.c | 6 +-- drivers/scsi/pm8001/pm8001_init.c | 79 ++++++++++++++++++++++++++++----------- drivers/scsi/pm8001/pm8001_sas.h | 2 +- drivers/scsi/pm8001/pm80xx_hwi.c | 28 ++++---------- 5 files changed, 72 insertions(+), 47 deletions(-) diff --git a/drivers/scsi/pm8001/pm8001_defs.h b/drivers/scsi/pm8001/pm8001_defs.h index 1bf1bcfaf010..501b574239e8 100644 --- a/drivers/scsi/pm8001/pm8001_defs.h +++ b/drivers/scsi/pm8001/pm8001_defs.h @@ -75,7 +75,7 @@ enum port_type { }; /* driver compile-time configuration */ -#define PM8001_MAX_CCB 256 /* max ccbs supported */ +#define PM8001_MAX_CCB 1024 /* max ccbs supported */ #define PM8001_MPI_QUEUE 1024 /* maximum mpi queue entries */ #define PM8001_MAX_INB_NUM 64 #define PM8001_MAX_OUTB_NUM 64 @@ -90,9 +90,11 @@ enum port_type { #define PM8001_MAX_PORTS 16 /* max. possible ports */ #define PM8001_MAX_DEVICES 2048 /* max supported device */ #define PM8001_MAX_MSIX_VEC 64 /* max msi-x int for spcv/ve */ +#define PM8001_RESERVE_SLOT 8 #define CONFIG_SCSI_PM8001_MAX_DMA_SG 528 #define PM8001_MAX_DMA_SG CONFIG_SCSI_PM8001_MAX_DMA_SG + enum memory_region_num { AAP1 = 0x0, /* application acceleration processor */ IOP, /* IO processor */ diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c index e9106575a30f..2b7b2954ec31 100644 --- a/drivers/scsi/pm8001/pm8001_hwi.c +++ b/drivers/scsi/pm8001/pm8001_hwi.c @@ -4375,8 +4375,7 @@ static int pm8001_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, /* fill in PRD (scatter/gather) table, if any */ if (task->num_scatter > 1) { pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; ssp_cmd.addr_low = cpu_to_le32(lower_32_bits(phys_addr)); ssp_cmd.addr_high = cpu_to_le32(upper_32_bits(phys_addr)); ssp_cmd.esgl = cpu_to_le32(1<<31); @@ -4449,8 +4448,7 @@ static int pm8001_chip_sata_req(struct pm8001_hba_info *pm8001_ha, /* fill in PRD (scatter/gather) table, if any */ if (task->num_scatter > 1) { pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; sata_cmd.addr_low = lower_32_bits(phys_addr); sata_cmd.addr_high = upper_32_bits(phys_addr); sata_cmd.esgl = cpu_to_le32(1 << 31); diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c index d6789a261c1c..d04d6ecd8c0f 100644 --- a/drivers/scsi/pm8001/pm8001_init.c +++ b/drivers/scsi/pm8001/pm8001_init.c @@ -56,6 +56,7 @@ MODULE_PARM_DESC(link_rate, "Enable link rate.\n" " 8: Link rate 12.0G\n"); static struct scsi_transport_template *pm8001_stt; +static int pm8001_init_ccb_tag(struct pm8001_hba_info *, struct Scsi_Host *, struct pci_dev *); /* * chip info structure to identify chip key functionality as @@ -302,9 +303,6 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, INIT_LIST_HEAD(&pm8001_ha->port[i].list); } - pm8001_ha->tags = kzalloc(PM8001_MAX_CCB, GFP_KERNEL); - if (!pm8001_ha->tags) - goto err_out; /* MPI Memory region 1 for AAP Event Log for fw */ pm8001_ha->memoryMap.region[AAP1].num_elements = 1; pm8001_ha->memoryMap.region[AAP1].element_size = PM8001_EVENT_LOG_SIZE; @@ -416,29 +414,11 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, pm8001_ha->devices[i].device_id = PM8001_MAX_DEVICES; pm8001_ha->devices[i].running_req = 0; } - /* Memory region for ccb_info*/ - pm8001_ha->ccb_info = kzalloc(PM8001_MAX_CCB - * sizeof(struct pm8001_ccb_info), GFP_KERNEL); - if (!pm8001_ha->ccb_info) { - rc = -ENOMEM; - goto err_out_noccb; - } - for (i = 0; i < PM8001_MAX_CCB; i++) { - pm8001_ha->ccb_info[i].ccb_dma_handle = - virt_to_phys(pm8001_ha->ccb_info) + - (i * sizeof(struct pm8001_ccb_info)); - pm8001_ha->ccb_info[i].task = NULL; - pm8001_ha->ccb_info[i].ccb_tag = 0xffffffff; - pm8001_ha->ccb_info[i].device = NULL; - ++pm8001_ha->tags_num; - } pm8001_ha->flags = PM8001F_INIT_TIME; /* Initialize tags */ pm8001_tag_init(pm8001_ha); return 0; -err_out_noccb: - kfree(pm8001_ha->devices); err_out_shost: scsi_remove_host(pm8001_ha->shost); err_out_nodev: @@ -1133,6 +1113,10 @@ static int pm8001_pci_probe(struct pci_dev *pdev, goto err_out_ha_free; } + rc = pm8001_init_ccb_tag(pm8001_ha, shost, pdev); + if (rc) + goto err_out_enable; + rc = scsi_add_host(shost, &pdev->dev); if (rc) goto err_out_ha_free; @@ -1178,6 +1162,59 @@ static int pm8001_pci_probe(struct pci_dev *pdev, return rc; } +/* + * pm8001_init_ccb_tag - allocate memory to CCB and tag. + * @pm8001_ha: our hba card information. + * @shost: scsi host which has been allocated outside. + */ +static int +pm8001_init_ccb_tag(struct pm8001_hba_info *pm8001_ha, struct Scsi_Host *shost, + struct pci_dev *pdev) +{ + int i, ret = 0; + u32 max_out_io, ccb_count; + u32 can_queue; + + max_out_io = pm8001_ha->main_cfg_tbl.pm80xx_tbl.max_out_io; + ccb_count = min_t(int, PM8001_MAX_CCB, max_out_io); + + /*Update to the scsi host*/ + can_queue = ccb_count - PM8001_RESERVE_SLOT; + shost->can_queue = can_queue; + + pm8001_ha->tags = kzalloc(ccb_count, GFP_KERNEL); + if (!pm8001_ha->tags) + goto err_out; + + /* Memory region for ccb_info*/ + pm8001_ha->ccb_info = (struct pm8001_ccb_info *) + kcalloc(ccb_count, sizeof(struct pm8001_ccb_info), GFP_KERNEL); + if (!pm8001_ha->ccb_info) { + ret = -ENOMEM; + goto err_out_noccb; + } + for (i = 0; i < ccb_count; i++) { + pm8001_ha->ccb_info[i].buf_prd = pci_alloc_consistent(pdev, + sizeof(struct pm8001_prd) * PM8001_MAX_DMA_SG, + &pm8001_ha->ccb_info[i].ccb_dma_handle); + if (!pm8001_ha->ccb_info[i].buf_prd) { + PM8001_FAIL_DBG(pm8001_ha, pm8001_printk + (KERN_ERR "pm80xx: ccb prd memory allocation error\n")); + goto err_out; + } + pm8001_ha->ccb_info[i].task = NULL; + pm8001_ha->ccb_info[i].ccb_tag = 0xffffffff; + pm8001_ha->ccb_info[i].device = NULL; + ++pm8001_ha->tags_num; + } + return 0; + +err_out_noccb: + kfree(pm8001_ha->devices); +err_out: + return -ENOMEM; +} + static void pm8001_pci_remove(struct pci_dev *pdev) { struct sas_ha_struct *sha = pci_get_drvdata(pdev); diff --git a/drivers/scsi/pm8001/pm8001_sas.h b/drivers/scsi/pm8001/pm8001_sas.h index bdfce3c3f619..9d7796a74ed4 100644 --- a/drivers/scsi/pm8001/pm8001_sas.h +++ b/drivers/scsi/pm8001/pm8001_sas.h @@ -315,7 +315,7 @@ struct pm8001_ccb_info { u32 ccb_tag; dma_addr_t ccb_dma_handle; struct pm8001_device *device; - struct pm8001_prd buf_prd[PM8001_MAX_DMA_SG]; + struct pm8001_prd *buf_prd; struct fw_control_ex *fw_control_context; u8 open_retry; }; diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c index 26e9e8877107..7593f248afb2 100644 --- a/drivers/scsi/pm8001/pm80xx_hwi.c +++ b/drivers/scsi/pm8001/pm80xx_hwi.c @@ -4483,8 +4483,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, if (task->num_scatter > 1) { pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; ssp_cmd.enc_addr_low = cpu_to_le32(lower_32_bits(phys_addr)); ssp_cmd.enc_addr_high = @@ -4513,9 +4512,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, end_addr_high, end_addr_low)); pm8001_chip_make_sg(task->scatter, 1, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, - buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; ssp_cmd.enc_addr_low = cpu_to_le32(lower_32_bits(phys_addr)); ssp_cmd.enc_addr_high = @@ -4543,8 +4540,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, if (task->num_scatter > 1) { pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; ssp_cmd.addr_low = cpu_to_le32(lower_32_bits(phys_addr)); ssp_cmd.addr_high = @@ -4572,9 +4568,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, end_addr_high, end_addr_low)); pm8001_chip_make_sg(task->scatter, 1, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, - buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; ssp_cmd.addr_low = cpu_to_le32(lower_32_bits(phys_addr)); ssp_cmd.addr_high = @@ -4666,8 +4660,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, if (task->num_scatter > 1) { pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; sata_cmd.enc_addr_low = lower_32_bits(phys_addr); sata_cmd.enc_addr_high = upper_32_bits(phys_addr); sata_cmd.enc_esgl = cpu_to_le32(1 << 31); @@ -4692,9 +4685,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, end_addr_high, end_addr_low)); pm8001_chip_make_sg(task->scatter, 1, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, - buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; sata_cmd.enc_addr_low = lower_32_bits(phys_addr); sata_cmd.enc_addr_high = @@ -4732,8 +4723,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, if (task->num_scatter > 1) { pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; sata_cmd.addr_low = lower_32_bits(phys_addr); sata_cmd.addr_high = upper_32_bits(phys_addr); sata_cmd.esgl = cpu_to_le32(1 << 31); @@ -4758,9 +4748,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, end_addr_high, end_addr_low)); pm8001_chip_make_sg(task->scatter, 1, ccb->buf_prd); - phys_addr = ccb->ccb_dma_handle + - offsetof(struct pm8001_ccb_info, - buf_prd[0]); + phys_addr = ccb->ccb_dma_handle; sata_cmd.addr_low = lower_32_bits(phys_addr); sata_cmd.addr_high = -- 2.16.3 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported 2020-09-25 6:16 ` [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported Viswas G @ 2020-09-25 8:16 ` kernel test robot 2020-09-25 9:56 ` John Garry 2020-10-05 11:21 ` Jinpu Wang 2 siblings, 0 replies; 16+ messages in thread From: kernel test robot @ 2020-09-25 8:16 UTC (permalink / raw) To: Viswas G, linux-scsi Cc: kbuild-all, Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang [-- Attachment #1: Type: text/plain, Size: 3646 bytes --] Hi Viswas, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on mkp-scsi/for-next] [also build test WARNING on scsi/for-next v5.9-rc6 next-20200924] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch] url: https://github.com/0day-ci/linux/commits/Viswas-G/pm80xx-updates/20200925-141508 base: https://git.kernel.org/pub/scm/linux/kernel/git/mkp/scsi.git for-next config: xtensa-allyesconfig (attached as .config) compiler: xtensa-linux-gcc (GCC) 9.3.0 reproduce (this is a W=1 build): wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # https://github.com/0day-ci/linux/commit/417584937b14cc5369025f0d5cd9882674b8a2f8 git remote add linux-review https://github.com/0day-ci/linux git fetch --no-tags linux-review Viswas-G/pm80xx-updates/20200925-141508 git checkout 417584937b14cc5369025f0d5cd9882674b8a2f8 # save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=xtensa If you fix the issue, kindly add following tag as appropriate Reported-by: kernel test robot <lkp@intel.com> All warnings (new ones prefixed by >>): drivers/scsi/pm8001/pm8001_init.c: In function 'pm8001_init_ccb_tag': >> drivers/scsi/pm8001/pm8001_init.c:1174:9: warning: variable 'ret' set but not used [-Wunused-but-set-variable] 1174 | int i, ret = 0; | ^~~ vim +/ret +1174 drivers/scsi/pm8001/pm8001_init.c 1164 1165 /* 1166 * pm8001_init_ccb_tag - allocate memory to CCB and tag. 1167 * @pm8001_ha: our hba card information. 1168 * @shost: scsi host which has been allocated outside. 1169 */ 1170 static int 1171 pm8001_init_ccb_tag(struct pm8001_hba_info *pm8001_ha, struct Scsi_Host *shost, 1172 struct pci_dev *pdev) 1173 { > 1174 int i, ret = 0; 1175 u32 max_out_io, ccb_count; 1176 u32 can_queue; 1177 1178 max_out_io = pm8001_ha->main_cfg_tbl.pm80xx_tbl.max_out_io; 1179 ccb_count = min_t(int, PM8001_MAX_CCB, max_out_io); 1180 1181 /*Update to the scsi host*/ 1182 can_queue = ccb_count - PM8001_RESERVE_SLOT; 1183 shost->can_queue = can_queue; 1184 1185 pm8001_ha->tags = kzalloc(ccb_count, GFP_KERNEL); 1186 if (!pm8001_ha->tags) 1187 goto err_out; 1188 1189 /* Memory region for ccb_info*/ 1190 pm8001_ha->ccb_info = (struct pm8001_ccb_info *) 1191 kcalloc(ccb_count, sizeof(struct pm8001_ccb_info), GFP_KERNEL); 1192 if (!pm8001_ha->ccb_info) { 1193 ret = -ENOMEM; 1194 goto err_out_noccb; 1195 } 1196 for (i = 0; i < ccb_count; i++) { 1197 pm8001_ha->ccb_info[i].buf_prd = pci_alloc_consistent(pdev, 1198 sizeof(struct pm8001_prd) * PM8001_MAX_DMA_SG, 1199 &pm8001_ha->ccb_info[i].ccb_dma_handle); 1200 if (!pm8001_ha->ccb_info[i].buf_prd) { 1201 PM8001_FAIL_DBG(pm8001_ha, pm8001_printk 1202 (KERN_ERR "pm80xx: ccb prd memory allocation error\n")); 1203 goto err_out; 1204 } 1205 pm8001_ha->ccb_info[i].task = NULL; 1206 pm8001_ha->ccb_info[i].ccb_tag = 0xffffffff; 1207 pm8001_ha->ccb_info[i].device = NULL; 1208 ++pm8001_ha->tags_num; 1209 } 1210 return 0; 1211 1212 err_out_noccb: 1213 kfree(pm8001_ha->devices); 1214 err_out: 1215 return -ENOMEM; 1216 } 1217 --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 65068 bytes --] ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported 2020-09-25 6:16 ` [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported Viswas G 2020-09-25 8:16 ` kernel test robot @ 2020-09-25 9:56 ` John Garry 2020-09-25 10:11 ` Jinpu Wang 2020-10-05 11:21 ` Jinpu Wang 2 siblings, 1 reply; 16+ messages in thread From: John Garry @ 2020-09-25 9:56 UTC (permalink / raw) To: Viswas G, linux-scsi Cc: Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang On 25/09/2020 07:16, Viswas G wrote: > From: Viswas G<Viswas.G@microchip.com> > > Increasing the number of Outstanding IOs from 256 to 1024. > CCB and tag are allocated according to outstanding IOs. > Also updating the can_queue value (max_out_io - PM8001_RESERVE_SLOT) > to scsi midlayer. > > Signed-off-by: Viswas G<Viswas.G@microchip.com> > Signed-off-by: Ruksar Devadi<Ruksar.devadi@microchip.com> Any reason you can't also use the request->tag (instead of generating tags internally) for added performance boost? Many other LLDDs do this, as managing tags has a performance overhead. Thanks, John ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported 2020-09-25 9:56 ` John Garry @ 2020-09-25 10:11 ` Jinpu Wang 2020-09-28 6:38 ` Viswas.G 0 siblings, 1 reply; 16+ messages in thread From: Jinpu Wang @ 2020-09-25 10:11 UTC (permalink / raw) To: John Garry Cc: Viswas G, Linux SCSI Mailinglist, Vasanthalakshmi.Tharmarajan, Viswas G, Ruksar.devadi On Fri, Sep 25, 2020 at 11:59 AM John Garry <john.garry@huawei.com> wrote: > > On 25/09/2020 07:16, Viswas G wrote: > > From: Viswas G<Viswas.G@microchip.com> > > > > Increasing the number of Outstanding IOs from 256 to 1024. > > CCB and tag are allocated according to outstanding IOs. > > Also updating the can_queue value (max_out_io - PM8001_RESERVE_SLOT) > > to scsi midlayer. > > > > Signed-off-by: Viswas G<Viswas.G@microchip.com> > > Signed-off-by: Ruksar Devadi<Ruksar.devadi@microchip.com> > > Any reason you can't also use the request->tag (instead of generating > tags internally) for added performance boost? Many other LLDDs do this, > as managing tags has a performance overhead. > > Thanks, > John +1, I think the reason probably is easily compatible with the older kernel. For upstream one, it makes sense to switch to request->tag. Thanks! ^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported 2020-09-25 10:11 ` Jinpu Wang @ 2020-09-28 6:38 ` Viswas.G 0 siblings, 0 replies; 16+ messages in thread From: Viswas.G @ 2020-09-28 6:38 UTC (permalink / raw) To: jinpu.wang, john.garry Cc: Viswas.G, linux-scsi, Vasanthalakshmi.Tharmarajan, Ruksar.devadi Thank you Jinpu and John for the review. We are yet to add this support in the driver to make use of the block layer request tag. We are planning to add that feature in our async driver first and see the performance improvements and do a full QA team validation before submitting to open source. The current patch set was created based on our async driver which has been tested by our QA team. We could see some performance boost with these changes as well. Regards, Viswas G > -----Original Message----- > From: Jinpu Wang <jinpu.wang@cloud.ionos.com> > Sent: Friday, September 25, 2020 3:42 PM > To: John Garry <john.garry@huawei.com> > Cc: Viswas G <Viswas.G@microchip.com.com>; Linux SCSI Mailinglist <linux- > scsi@vger.kernel.org>; Vasanthalakshmi Tharmarajan - I30664 > <Vasanthalakshmi.Tharmarajan@microchip.com>; Viswas G - I30667 > <Viswas.G@microchip.com>; Ruksar Devadi - I52327 > <Ruksar.devadi@microchip.com> > Subject: Re: [PATCH 3/4] pm80xx : Increase the number of outstanding IO > supported > > EXTERNAL EMAIL: Do not click links or open attachments unless you know > the content is safe > > On Fri, Sep 25, 2020 at 11:59 AM John Garry <john.garry@huawei.com> > wrote: > > > > On 25/09/2020 07:16, Viswas G wrote: > > > From: Viswas G<Viswas.G@microchip.com> > > > > > > Increasing the number of Outstanding IOs from 256 to 1024. > > > CCB and tag are allocated according to outstanding IOs. > > > Also updating the can_queue value (max_out_io - > PM8001_RESERVE_SLOT) > > > to scsi midlayer. > > > > > > Signed-off-by: Viswas G<Viswas.G@microchip.com> > > > Signed-off-by: Ruksar Devadi<Ruksar.devadi@microchip.com> > > > > Any reason you can't also use the request->tag (instead of generating > > tags internally) for added performance boost? Many other LLDDs do > > this, as managing tags has a performance overhead. > > > > Thanks, > > John > > +1, I think the reason probably is easily compatible with the older kernel. > For upstream one, it makes sense to switch to request->tag. > > Thanks! ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported 2020-09-25 6:16 ` [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported Viswas G 2020-09-25 8:16 ` kernel test robot 2020-09-25 9:56 ` John Garry @ 2020-10-05 11:21 ` Jinpu Wang 2 siblings, 0 replies; 16+ messages in thread From: Jinpu Wang @ 2020-10-05 11:21 UTC (permalink / raw) To: Viswas G Cc: Linux SCSI Mailinglist, Vasanthalakshmi.Tharmarajan, Viswas G, Ruksar.devadi On Fri, Sep 25, 2020 at 8:06 AM Viswas G <Viswas.G@microchip.com.com> wrote: > > From: Viswas G <Viswas.G@microchip.com> > > Increasing the number of Outstanding IOs from 256 to 1024. > CCB and tag are allocated according to outstanding IOs. > Also updating the can_queue value (max_out_io - PM8001_RESERVE_SLOT) > to scsi midlayer. > > Signed-off-by: Viswas G <Viswas.G@microchip.com> > Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> As commented in other patches, please also document the reason for the change in commit message. The patch itself looks fine to me! Thank you > --- > drivers/scsi/pm8001/pm8001_defs.h | 4 +- > drivers/scsi/pm8001/pm8001_hwi.c | 6 +-- > drivers/scsi/pm8001/pm8001_init.c | 79 ++++++++++++++++++++++++++++----------- > drivers/scsi/pm8001/pm8001_sas.h | 2 +- > drivers/scsi/pm8001/pm80xx_hwi.c | 28 ++++---------- > 5 files changed, 72 insertions(+), 47 deletions(-) > > diff --git a/drivers/scsi/pm8001/pm8001_defs.h b/drivers/scsi/pm8001/pm8001_defs.h > index 1bf1bcfaf010..501b574239e8 100644 > --- a/drivers/scsi/pm8001/pm8001_defs.h > +++ b/drivers/scsi/pm8001/pm8001_defs.h > @@ -75,7 +75,7 @@ enum port_type { > }; > > /* driver compile-time configuration */ > -#define PM8001_MAX_CCB 256 /* max ccbs supported */ > +#define PM8001_MAX_CCB 1024 /* max ccbs supported */ > #define PM8001_MPI_QUEUE 1024 /* maximum mpi queue entries */ > #define PM8001_MAX_INB_NUM 64 > #define PM8001_MAX_OUTB_NUM 64 > @@ -90,9 +90,11 @@ enum port_type { > #define PM8001_MAX_PORTS 16 /* max. possible ports */ > #define PM8001_MAX_DEVICES 2048 /* max supported device */ > #define PM8001_MAX_MSIX_VEC 64 /* max msi-x int for spcv/ve */ > +#define PM8001_RESERVE_SLOT 8 > > #define CONFIG_SCSI_PM8001_MAX_DMA_SG 528 > #define PM8001_MAX_DMA_SG CONFIG_SCSI_PM8001_MAX_DMA_SG > + > enum memory_region_num { > AAP1 = 0x0, /* application acceleration processor */ > IOP, /* IO processor */ > diff --git a/drivers/scsi/pm8001/pm8001_hwi.c b/drivers/scsi/pm8001/pm8001_hwi.c > index e9106575a30f..2b7b2954ec31 100644 > --- a/drivers/scsi/pm8001/pm8001_hwi.c > +++ b/drivers/scsi/pm8001/pm8001_hwi.c > @@ -4375,8 +4375,7 @@ static int pm8001_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > /* fill in PRD (scatter/gather) table, if any */ > if (task->num_scatter > 1) { > pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > ssp_cmd.addr_low = cpu_to_le32(lower_32_bits(phys_addr)); > ssp_cmd.addr_high = cpu_to_le32(upper_32_bits(phys_addr)); > ssp_cmd.esgl = cpu_to_le32(1<<31); > @@ -4449,8 +4448,7 @@ static int pm8001_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > /* fill in PRD (scatter/gather) table, if any */ > if (task->num_scatter > 1) { > pm8001_chip_make_sg(task->scatter, ccb->n_elem, ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > sata_cmd.addr_low = lower_32_bits(phys_addr); > sata_cmd.addr_high = upper_32_bits(phys_addr); > sata_cmd.esgl = cpu_to_le32(1 << 31); > diff --git a/drivers/scsi/pm8001/pm8001_init.c b/drivers/scsi/pm8001/pm8001_init.c > index d6789a261c1c..d04d6ecd8c0f 100644 > --- a/drivers/scsi/pm8001/pm8001_init.c > +++ b/drivers/scsi/pm8001/pm8001_init.c > @@ -56,6 +56,7 @@ MODULE_PARM_DESC(link_rate, "Enable link rate.\n" > " 8: Link rate 12.0G\n"); > > static struct scsi_transport_template *pm8001_stt; > +static int pm8001_init_ccb_tag(struct pm8001_hba_info *, struct Scsi_Host *, struct pci_dev *); > > /* > * chip info structure to identify chip key functionality as > @@ -302,9 +303,6 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > INIT_LIST_HEAD(&pm8001_ha->port[i].list); > } > > - pm8001_ha->tags = kzalloc(PM8001_MAX_CCB, GFP_KERNEL); > - if (!pm8001_ha->tags) > - goto err_out; > /* MPI Memory region 1 for AAP Event Log for fw */ > pm8001_ha->memoryMap.region[AAP1].num_elements = 1; > pm8001_ha->memoryMap.region[AAP1].element_size = PM8001_EVENT_LOG_SIZE; > @@ -416,29 +414,11 @@ static int pm8001_alloc(struct pm8001_hba_info *pm8001_ha, > pm8001_ha->devices[i].device_id = PM8001_MAX_DEVICES; > pm8001_ha->devices[i].running_req = 0; > } > - /* Memory region for ccb_info*/ > - pm8001_ha->ccb_info = kzalloc(PM8001_MAX_CCB > - * sizeof(struct pm8001_ccb_info), GFP_KERNEL); > - if (!pm8001_ha->ccb_info) { > - rc = -ENOMEM; > - goto err_out_noccb; > - } > - for (i = 0; i < PM8001_MAX_CCB; i++) { > - pm8001_ha->ccb_info[i].ccb_dma_handle = > - virt_to_phys(pm8001_ha->ccb_info) + > - (i * sizeof(struct pm8001_ccb_info)); > - pm8001_ha->ccb_info[i].task = NULL; > - pm8001_ha->ccb_info[i].ccb_tag = 0xffffffff; > - pm8001_ha->ccb_info[i].device = NULL; > - ++pm8001_ha->tags_num; > - } > pm8001_ha->flags = PM8001F_INIT_TIME; > /* Initialize tags */ > pm8001_tag_init(pm8001_ha); > return 0; > > -err_out_noccb: > - kfree(pm8001_ha->devices); > err_out_shost: > scsi_remove_host(pm8001_ha->shost); > err_out_nodev: > @@ -1133,6 +1113,10 @@ static int pm8001_pci_probe(struct pci_dev *pdev, > goto err_out_ha_free; > } > > + rc = pm8001_init_ccb_tag(pm8001_ha, shost, pdev); > + if (rc) > + goto err_out_enable; > + > rc = scsi_add_host(shost, &pdev->dev); > if (rc) > goto err_out_ha_free; > @@ -1178,6 +1162,59 @@ static int pm8001_pci_probe(struct pci_dev *pdev, > return rc; > } > > +/* > + * pm8001_init_ccb_tag - allocate memory to CCB and tag. > + * @pm8001_ha: our hba card information. > + * @shost: scsi host which has been allocated outside. > + */ > +static int > +pm8001_init_ccb_tag(struct pm8001_hba_info *pm8001_ha, struct Scsi_Host *shost, > + struct pci_dev *pdev) > +{ > + int i, ret = 0; > + u32 max_out_io, ccb_count; > + u32 can_queue; > + > + max_out_io = pm8001_ha->main_cfg_tbl.pm80xx_tbl.max_out_io; > + ccb_count = min_t(int, PM8001_MAX_CCB, max_out_io); > + > + /*Update to the scsi host*/ > + can_queue = ccb_count - PM8001_RESERVE_SLOT; > + shost->can_queue = can_queue; > + > + pm8001_ha->tags = kzalloc(ccb_count, GFP_KERNEL); > + if (!pm8001_ha->tags) > + goto err_out; > + > + /* Memory region for ccb_info*/ > + pm8001_ha->ccb_info = (struct pm8001_ccb_info *) > + kcalloc(ccb_count, sizeof(struct pm8001_ccb_info), GFP_KERNEL); > + if (!pm8001_ha->ccb_info) { > + ret = -ENOMEM; > + goto err_out_noccb; > + } > + for (i = 0; i < ccb_count; i++) { > + pm8001_ha->ccb_info[i].buf_prd = pci_alloc_consistent(pdev, > + sizeof(struct pm8001_prd) * PM8001_MAX_DMA_SG, > + &pm8001_ha->ccb_info[i].ccb_dma_handle); > + if (!pm8001_ha->ccb_info[i].buf_prd) { > + PM8001_FAIL_DBG(pm8001_ha, pm8001_printk > + (KERN_ERR "pm80xx: ccb prd memory allocation error\n")); > + goto err_out; > + } > + pm8001_ha->ccb_info[i].task = NULL; > + pm8001_ha->ccb_info[i].ccb_tag = 0xffffffff; > + pm8001_ha->ccb_info[i].device = NULL; > + ++pm8001_ha->tags_num; > + } > + return 0; > + > +err_out_noccb: > + kfree(pm8001_ha->devices); > +err_out: > + return -ENOMEM; > +} > + > static void pm8001_pci_remove(struct pci_dev *pdev) > { > struct sas_ha_struct *sha = pci_get_drvdata(pdev); > diff --git a/drivers/scsi/pm8001/pm8001_sas.h b/drivers/scsi/pm8001/pm8001_sas.h > index bdfce3c3f619..9d7796a74ed4 100644 > --- a/drivers/scsi/pm8001/pm8001_sas.h > +++ b/drivers/scsi/pm8001/pm8001_sas.h > @@ -315,7 +315,7 @@ struct pm8001_ccb_info { > u32 ccb_tag; > dma_addr_t ccb_dma_handle; > struct pm8001_device *device; > - struct pm8001_prd buf_prd[PM8001_MAX_DMA_SG]; > + struct pm8001_prd *buf_prd; > struct fw_control_ex *fw_control_context; > u8 open_retry; > }; > diff --git a/drivers/scsi/pm8001/pm80xx_hwi.c b/drivers/scsi/pm8001/pm80xx_hwi.c > index 26e9e8877107..7593f248afb2 100644 > --- a/drivers/scsi/pm8001/pm80xx_hwi.c > +++ b/drivers/scsi/pm8001/pm80xx_hwi.c > @@ -4483,8 +4483,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > if (task->num_scatter > 1) { > pm8001_chip_make_sg(task->scatter, > ccb->n_elem, ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > ssp_cmd.enc_addr_low = > cpu_to_le32(lower_32_bits(phys_addr)); > ssp_cmd.enc_addr_high = > @@ -4513,9 +4512,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > end_addr_high, end_addr_low)); > pm8001_chip_make_sg(task->scatter, 1, > ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, > - buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > ssp_cmd.enc_addr_low = > cpu_to_le32(lower_32_bits(phys_addr)); > ssp_cmd.enc_addr_high = > @@ -4543,8 +4540,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > if (task->num_scatter > 1) { > pm8001_chip_make_sg(task->scatter, ccb->n_elem, > ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > ssp_cmd.addr_low = > cpu_to_le32(lower_32_bits(phys_addr)); > ssp_cmd.addr_high = > @@ -4572,9 +4568,7 @@ static int pm80xx_chip_ssp_io_req(struct pm8001_hba_info *pm8001_ha, > end_addr_high, end_addr_low)); > pm8001_chip_make_sg(task->scatter, 1, > ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, > - buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > ssp_cmd.addr_low = > cpu_to_le32(lower_32_bits(phys_addr)); > ssp_cmd.addr_high = > @@ -4666,8 +4660,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > if (task->num_scatter > 1) { > pm8001_chip_make_sg(task->scatter, > ccb->n_elem, ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > sata_cmd.enc_addr_low = lower_32_bits(phys_addr); > sata_cmd.enc_addr_high = upper_32_bits(phys_addr); > sata_cmd.enc_esgl = cpu_to_le32(1 << 31); > @@ -4692,9 +4685,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > end_addr_high, end_addr_low)); > pm8001_chip_make_sg(task->scatter, 1, > ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, > - buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > sata_cmd.enc_addr_low = > lower_32_bits(phys_addr); > sata_cmd.enc_addr_high = > @@ -4732,8 +4723,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > if (task->num_scatter > 1) { > pm8001_chip_make_sg(task->scatter, > ccb->n_elem, ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > sata_cmd.addr_low = lower_32_bits(phys_addr); > sata_cmd.addr_high = upper_32_bits(phys_addr); > sata_cmd.esgl = cpu_to_le32(1 << 31); > @@ -4758,9 +4748,7 @@ static int pm80xx_chip_sata_req(struct pm8001_hba_info *pm8001_ha, > end_addr_high, end_addr_low)); > pm8001_chip_make_sg(task->scatter, 1, > ccb->buf_prd); > - phys_addr = ccb->ccb_dma_handle + > - offsetof(struct pm8001_ccb_info, > - buf_prd[0]); > + phys_addr = ccb->ccb_dma_handle; > sata_cmd.addr_low = > lower_32_bits(phys_addr); > sata_cmd.addr_high = > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH 4/4] pm80xx : Driver version update 2020-09-25 6:16 [PATCH 0/4] pm80xx updates Viswas G ` (3 preceding siblings ...) 2020-09-25 6:16 ` [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported Viswas G @ 2020-09-25 6:16 ` Viswas G 2020-10-05 11:18 ` Jinpu Wang 4 siblings, 1 reply; 16+ messages in thread From: Viswas G @ 2020-09-25 6:16 UTC (permalink / raw) To: linux-scsi Cc: Vasanthalakshmi.Tharmarajan, Viswas.G, Ruksar.devadi, Jinpu Wang From: Viswas G <Viswas.G@microchip.com> Update driver version from "0.1.39" -> "0.1.40" Signed-off-by: Viswas G <Viswas.G@microchip.com> Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> --- drivers/scsi/pm8001/pm8001_sas.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/pm8001/pm8001_sas.h b/drivers/scsi/pm8001/pm8001_sas.h index 9d7796a74ed4..95663e138083 100644 --- a/drivers/scsi/pm8001/pm8001_sas.h +++ b/drivers/scsi/pm8001/pm8001_sas.h @@ -58,7 +58,7 @@ #include "pm8001_defs.h" #define DRV_NAME "pm80xx" -#define DRV_VERSION "0.1.39" +#define DRV_VERSION "0.1.40" #define PM8001_FAIL_LOGGING 0x01 /* Error message logging */ #define PM8001_INIT_LOGGING 0x02 /* driver init logging */ #define PM8001_DISC_LOGGING 0x04 /* discovery layer logging */ -- 2.16.3 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH 4/4] pm80xx : Driver version update 2020-09-25 6:16 ` [PATCH 4/4] pm80xx : Driver version update Viswas G @ 2020-10-05 11:18 ` Jinpu Wang 0 siblings, 0 replies; 16+ messages in thread From: Jinpu Wang @ 2020-10-05 11:18 UTC (permalink / raw) To: Viswas G Cc: Linux SCSI Mailinglist, Vasanthalakshmi.Tharmarajan, Viswas G, Ruksar.devadi On Fri, Sep 25, 2020 at 8:06 AM Viswas G <Viswas.G@microchip.com.com> wrote: > > From: Viswas G <Viswas.G@microchip.com> > > Update driver version from "0.1.39" -> "0.1.40" > > Signed-off-by: Viswas G <Viswas.G@microchip.com> > Signed-off-by: Ruksar Devadi <Ruksar.devadi@microchip.com> Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com> > --- > drivers/scsi/pm8001/pm8001_sas.h | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/scsi/pm8001/pm8001_sas.h b/drivers/scsi/pm8001/pm8001_sas.h > index 9d7796a74ed4..95663e138083 100644 > --- a/drivers/scsi/pm8001/pm8001_sas.h > +++ b/drivers/scsi/pm8001/pm8001_sas.h > @@ -58,7 +58,7 @@ > #include "pm8001_defs.h" > > #define DRV_NAME "pm80xx" > -#define DRV_VERSION "0.1.39" > +#define DRV_VERSION "0.1.40" > #define PM8001_FAIL_LOGGING 0x01 /* Error message logging */ > #define PM8001_INIT_LOGGING 0x02 /* driver init logging */ > #define PM8001_DISC_LOGGING 0x04 /* discovery layer logging */ > -- > 2.16.3 > ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2020-10-05 11:21 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-09-25 6:16 [PATCH 0/4] pm80xx updates Viswas G 2020-09-25 6:13 ` Jinpu Wang 2020-09-25 6:36 ` Viswas.G 2020-09-25 6:16 ` [PATCH 1/4] pm80xx : Increase number of supported queues Viswas G 2020-09-25 9:14 ` John Garry 2020-10-05 11:13 ` Jinpu Wang 2020-09-25 6:16 ` [PATCH 2/4] pm80xx : Remove DMA memory allocation for ccb and device Viswas G 2020-10-05 11:16 ` Jinpu Wang 2020-09-25 6:16 ` [PATCH 3/4] pm80xx : Increase the number of outstanding IO supported Viswas G 2020-09-25 8:16 ` kernel test robot 2020-09-25 9:56 ` John Garry 2020-09-25 10:11 ` Jinpu Wang 2020-09-28 6:38 ` Viswas.G 2020-10-05 11:21 ` Jinpu Wang 2020-09-25 6:16 ` [PATCH 4/4] pm80xx : Driver version update Viswas G 2020-10-05 11:18 ` Jinpu Wang
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox