[PATCH][SDK v1.2] sata: I/O load balancing

[PATCH][SDK v1.2] sata: I/O load balancing

[Qiang Liu]

From: Qiang Liu <b32616@freescale.com>

Reduce interrupt singnals through reset Interrupt Coalescing Control Reg.
Increase the threshold value of interrupt and timer will reduce the number
of complete interrupt sharply. Improve the system performance effectively.

Signed-off-by: Qiang Liu <qiang.liu@freescale.net>
---
Description:
  1. sata-fsl interrupt will be raised 130 thousand times when write 8G file
(dd if=/dev/zero of=/dev/sda2 bs=128K count=65536);
  2. most of interrupts raised because of only 1-4 commands completed;
  3. only 30 thousand times will be raised after set max interrupt threshold,
more interrupts are coalesced as the description of ICC;

Performance Improvement:
  use top command,
  [root@p2020ds root]# dd if=/dev/zero of=/dev/sda2 bs=128K count=65536 &
  [root@p2020ds root]# top

   CPU %  |  dd   |  flush-8:0 | softirq
  ---------------------------------------
   before | 20-22 |    17-19   |    7
  ---------------------------------------
   after  | 18-21 |    15-16   |    5
  ---------------------------------------

 drivers/ata/sata_fsl.c |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/drivers/ata/sata_fsl.c b/drivers/ata/sata_fsl.c
index 3547000..93f8b00 100644
--- a/drivers/ata/sata_fsl.c
+++ b/drivers/ata/sata_fsl.c
@@ -6,7 +6,7 @@
  * Author: Ashish Kalra <ashish.kalra@freescale.com>
  * Li Yang <leoli@freescale.com>
  *
- * Copyright (c) 2006-2007, 2011 Freescale Semiconductor, Inc.
+ * Copyright (c) 2006-2007, 2011-2012 Freescale Semiconductor, Inc.
  *
  * This program is free software; you can redistribute  it and/or modify it
  * under  the terms of  the GNU General  Public License as published by the
@@ -162,6 +162,16 @@ enum {
 };

 /*
+ * Interrupt Coalescing Control Register bitdefs
+ */
+enum {
+	ICC_MIN_INT_THRESHOLD_COUNT = (1 << 24),
+	ICC_MAX_INT_THRESHOLD_COUNT = (((1 << 4) - 1) << 24),
+	ICC_MIN_INT_THRESHOLD_TIMER = 1,
+	ICC_MAX_INT_THRESHOLD_TIMER = ((1 << 18) - 1),
+};
+
+/*
  * SATA Superset Registers
  */
 enum {
@@ -460,6 +470,13 @@ static unsigned int sata_fsl_qc_issue(struct ata_queued_cmd *qc)
 	/* Simply queue command to the controller/device */
 	iowrite32(1 << tag, CQ + hcr_base);

+	/*
+	 * reset the number of command complete bits which will cause the
+	 * interrupt to be signaled
+	 */
+	iowrite32(ICC_MAX_INT_THRESHOLD_COUNT | ICC_MAX_INT_THRESHOLD_TIMER,
+			ICC + hcr_base);
+
 	VPRINTK("xx_qc_issue called, tag=%d, CQ=0x%x, CA=0x%x\n",
 		tag, ioread32(CQ + hcr_base), ioread32(CA + hcr_base));

--
1.6.4

[PATCH][SDK v1.2] sata: I/O load balancing

[Qiang Liu]

From: Qiang Liu <b32616@freescale.com>

Reduce interrupt singnals through reset Interrupt Coalescing Control Reg.
Increase the threshold value of interrupt and timer will reduce the number
of complete interrupt sharply. Improve the system performance effectively.

Signed-off-by: Qiang Liu <qiang.liu@freescale.com>
---
Description:
  1. sata-fsl interrupt will be raised 130 thousand times when write 8G file
(dd if=/dev/zero of=/dev/sda2 bs=128K count=65536);
  2. most of interrupts raised because of only 1-4 commands completed;
  3. only 30 thousand times will be raised after set max interrupt threshold,
more interrupts are coalesced as the description of ICC;

Performance Improvement:
  use top command,
  [root@p2020ds root]# dd if=/dev/zero of=/dev/sda2 bs=128K count=65536 &
  [root@p2020ds root]# top

   CPU %  |  dd   |  flush-8:0 | softirq
  ---------------------------------------
   before | 20-22 |    17-19   |    7
  ---------------------------------------
   after  | 18-21 |    15-16   |    5
  ---------------------------------------

 drivers/ata/sata_fsl.c |   19 ++++++++++++++++++-
 1 files changed, 18 insertions(+), 1 deletions(-)

diff --git a/drivers/ata/sata_fsl.c b/drivers/ata/sata_fsl.c
index 3547000..93f8b00 100644
--- a/drivers/ata/sata_fsl.c
+++ b/drivers/ata/sata_fsl.c
@@ -6,7 +6,7 @@
  * Author: Ashish Kalra <ashish.kalra@freescale.com>
  * Li Yang <leoli@freescale.com>
  *
- * Copyright (c) 2006-2007, 2011 Freescale Semiconductor, Inc.
+ * Copyright (c) 2006-2007, 2011-2012 Freescale Semiconductor, Inc.
  *
  * This program is free software; you can redistribute  it and/or modify it
  * under  the terms of  the GNU General  Public License as published by the
@@ -162,6 +162,16 @@ enum {
 };

 /*
+ * Interrupt Coalescing Control Register bitdefs
+ */
+enum {
+	ICC_MIN_INT_THRESHOLD_COUNT = (1 << 24),
+	ICC_MAX_INT_THRESHOLD_COUNT = (((1 << 4) - 1) << 24),
+	ICC_MIN_INT_THRESHOLD_TIMER = 1,
+	ICC_MAX_INT_THRESHOLD_TIMER = ((1 << 18) - 1),
+};
+
+/*
  * SATA Superset Registers
  */
 enum {
@@ -460,6 +470,13 @@ static unsigned int sata_fsl_qc_issue(struct ata_queued_cmd *qc)
 	/* Simply queue command to the controller/device */
 	iowrite32(1 << tag, CQ + hcr_base);

+	/*
+	 * reset the number of command complete bits which will cause the
+	 * interrupt to be signaled
+	 */
+	iowrite32(ICC_MAX_INT_THRESHOLD_COUNT | ICC_MAX_INT_THRESHOLD_TIMER,
+			ICC + hcr_base);
+
 	VPRINTK("xx_qc_issue called, tag=%d, CQ=0x%x, CA=0x%x\n",
 		tag, ioread32(CQ + hcr_base), ioread32(CA + hcr_base));

--
1.6.4

Re: [PATCH][SDK v1.2] sata: I/O load balancing

[Li Yang]

On Fri, Jan 13, 2012 at 4:25 PM, Qiang Liu <qiang.liu@freescale.com> wrote:
> From: Qiang Liu <b32616@freescale.com>
>
> Reduce interrupt singnals through reset Interrupt Coalescing Control Reg.
> Increase the threshold value of interrupt and timer will reduce the number
> of complete interrupt sharply. Improve the system performance effectively.

There is always a trade off of throughput and latency by using
interrupt coalescing.  It's not reasonable to assume that the
throughput is the only factor to be considered and set the coalescing
threshold and timeout to the max value by default.  Have you carried
out other benchmark like copying many small files?

It will be safer to make the coalescing runtime configurable like the
sata_mv driver, IMO.

- Leo

RE: [PATCH][SDK v1.2] sata: I/O load balancing

[Liu Qiang-B32616]

=0A=
________________________________________=0A=
> From: linux-ide-owner@vger.kernel.org [linux-ide-owner@vger.kernel.org] o=
n behalf of Li Yang [leoli@freescale.com]=0A=
> Sent: Friday, January 13, 2012 3:36 AM=0A=
> To: Liu Qiang-B32616=0A=
> Cc: jgarzik@pobox.com; linux-ide@vger.kernel.org; linux-kernel@vger.kerne=
l.org; linuxppc-dev@lists.ozlabs.org; Liu Qiang-B32616=0A=
> Subject: Re: [PATCH][SDK v1.2] sata: I/O load balancing=0A=
=0A=
> On Fri, Jan 13, 2012 at 4:25 PM, Qiang Liu <qiang.liu@freescale.com> wrot=
e:=0A=
> > From: Qiang Liu <b32616@freescale.com>=0A=
> >=0A=
> > Reduce interrupt singnals through reset Interrupt Coalescing Control Re=
g.=0A=
> > Increase the threshold value of interrupt and timer will reduce the num=
ber=0A=
> > of complete interrupt sharply. Improve the system performance effective=
ly.=0A=
=0A=
> There is always a trade off of throughput and latency by using=0A=
> interrupt coalescing.  It's not reasonable to assume that the=0A=
> throughput is the only factor to be considered and set the coalescing=0A=
> threshold and timeout to the max value by default.  Have you carried=0A=
> out other benchmark like copying many small files?=0A=
No, I didn't test small file. I think this won't affect system load. I can =
have a test=0A=
and describe the result in next patch.=0A=
=0A=
> It will be safer to make the coalescing runtime configurable like the=0A=
> sata_mv driver, IMO.=0A=
Ok, I will do it like this in next patch.=0A=
=0A=
=0A=
- Leo=0A=
--=0A=
To unsubscribe from this list: send the line "unsubscribe linux-ide" in=0A=
the body of a message to majordomo@vger.kernel.org=0A=
More majordomo info at  http://vger.kernel.org/majordomo-info.html=0A=
=0A=

Re: [PATCH][SDK v1.2] sata: I/O load balancing

[Jeff Garzik]

On 01/13/2012 04:57 AM, Liu Qiang-B32616 wrote:
> No, I didn't test small file. I think this won't affect system load. I can have a test
> and describe the result in next patch.

Please do...