public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
@ 2005-09-14 20:01 Phil Dier
  2005-09-14 20:55 ` John Stoffel
  2005-09-14 21:41 ` Mark Lord
  0 siblings, 2 replies; 5+ messages in thread
From: Phil Dier @ 2005-09-14 20:01 UTC (permalink / raw)
  To: linux-kernel; +Cc: ziggy, Jack Massari, Scott Holdren

Hi,

I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
I have described before[0]. It still does. Briefly, I have a dual Xeon
2.8 with 4GB ram, 1 adaptec and 1 fusion-mpt scsi device, each connected
to 1 SCSI JBOD with 5 disks apiece[1]. I/O on scsi ids 0 and 1 both
work flawlessly. I built an md device across ids 2 and 3, and the disks
connected to the fusion-mpt are unbearably slow. This behaviour was not
present in 2.6.12.3.  When I partition the devices in question and mount
a filesystem on them, they exhibit the same behaviour individually.

[0]: http://www.ussg.iu.edu/hypermail/linux/kernel/0508.3/1233.html
[1]: http://www.icglink.com/debug-2.6.14-rc1.html


Here is what iostat looks like during what should be heavy I/O (my
tesh.sh script described in [1] is running), showing only the relevant
devices:


$ iostat 1 -k | awk '/(Device|md5|md6|md7)/{print $0;}'
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.62         0.00         2.46        244     198312
md6               0.62         0.00         2.46        228     198312
md7               0.61         0.01         2.45        440     197856
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5              25.74         0.00       102.97          0        104
md6              25.74         0.00       102.97          0        104
md7               9.90         0.00        39.60          0         40
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5              59.41         0.00       237.62          0        240
md6              60.40         3.96       237.62          4        240
md7              20.79         3.96        79.21          4         80
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5             100.99         0.00       403.96          0        408
md6             100.99         0.00       403.96          0        408
md7              72.28         0.00       289.11          0        292
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5           16242.57         0.00     64970.30          0      65620
md6           16242.57         0.00     64970.30          0      65620
md7           16230.69         0.00     64922.77          0      65572
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5            1014.85         0.00      4059.41          0       4100
md6            1014.85         0.00      4059.41          0       4100
md7            1014.85         0.00      4059.41          0       4100
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5            3298.02         0.00     13192.08          0      13324
md6            3298.02         0.00     13192.08          0      13324
md7            3298.02         0.00     13192.08          0      13324
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5            2789.11         0.00     11156.44          0      11268
md6            2789.11         0.00     11156.44          0      11268
md7            2789.11         0.00     11156.44          0      11268
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0
Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
md5               0.00         0.00         0.00          0          0
md6               0.00         0.00         0.00          0          0
md7               0.00         0.00         0.00          0          0


Thanks for looking. Please cc me on replies.

-- 

Phil Dier (ICGLink.com -- 615 370-1530 x733)

/* vim:set noai nocindent ts=8 sw=8: */

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
  2005-09-14 20:01 Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD Phil Dier
@ 2005-09-14 20:55 ` John Stoffel
  2005-09-14 21:41 ` Mark Lord
  1 sibling, 0 replies; 5+ messages in thread
From: John Stoffel @ 2005-09-14 20:55 UTC (permalink / raw)
  To: Phil Dier; +Cc: linux-kernel, ziggy, Jack Massari, Scott Holdren


Phil,

What happens if you pull 1gb or 2gb of memory from your box?  Running
a news feed shouldn't be memory intensive and maybe you've run into a
low-mem vs. high-mem issue.  It would be an interesting test in any
case.

John

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
  2005-09-14 20:01 Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD Phil Dier
  2005-09-14 20:55 ` John Stoffel
@ 2005-09-14 21:41 ` Mark Lord
  2005-09-14 21:58   ` Phil Dier
  2005-09-14 22:08   ` Phil Dier
  1 sibling, 2 replies; 5+ messages in thread
From: Mark Lord @ 2005-09-14 21:41 UTC (permalink / raw)
  To: Phil Dier; +Cc: linux-kernel, ziggy, Jack Massari, Scott Holdren

Phil Dier wrote:
> Hi,
> 
> I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
> I have described before[0]. It still does. Briefly, I have a dual Xeon
..

Do you still have HZ set to 1000 in your .config file? (as per 2.6.12)

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
  2005-09-14 21:41 ` Mark Lord
@ 2005-09-14 21:58   ` Phil Dier
  2005-09-14 22:08   ` Phil Dier
  1 sibling, 0 replies; 5+ messages in thread
From: Phil Dier @ 2005-09-14 21:58 UTC (permalink / raw)
  To: linux-kernel; +Cc: ziggy, jack, scott

On Wed, 14 Sep 2005 17:41:58 -0400
Mark Lord <lkml@rtr.ca> wrote:

> Phil Dier wrote:
> > Hi,
> > 
> > I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
> > I have described before[0]. It still does. Briefly, I have a dual Xeon
> ..
> 
> Do you still have HZ set to 1000 in your .config file? (as per 2.6.12)


No, the config for .14 is using the default of 100.  I'm recompiling with
HZ=1000 to see if that makes a difference.

-- 

Phil Dier (ICGLink.com -- 615 370-1530 x733)

/* vim:set noai nocindent ts=8 sw=8: */

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
  2005-09-14 21:41 ` Mark Lord
  2005-09-14 21:58   ` Phil Dier
@ 2005-09-14 22:08   ` Phil Dier
  1 sibling, 0 replies; 5+ messages in thread
From: Phil Dier @ 2005-09-14 22:08 UTC (permalink / raw)
  To: linux-kernel; +Cc: ziggy, jack, scott

On Wed, 14 Sep 2005 17:41:58 -0400
Mark Lord <lkml@rtr.ca> wrote:

> Phil Dier wrote:
> > Hi,
> > 
> > I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
> > I have described before[0]. It still does. Briefly, I have a dual Xeon
> ..
> 
> Do you still have HZ set to 1000 in your .config file? (as per 2.6.12)


With HZ at 1000, the speeds seem to be fine.  Why would this occur with
just this one controller?  Just driver differences?


-- 

Phil Dier (ICGLink.com -- 615 370-1530 x733)

/* vim:set noai nocindent ts=8 sw=8: */

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2005-09-14 22:08 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-09-14 20:01 Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD Phil Dier
2005-09-14 20:55 ` John Stoffel
2005-09-14 21:41 ` Mark Lord
2005-09-14 21:58   ` Phil Dier
2005-09-14 22:08   ` Phil Dier

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox