* Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
@ 2005-09-14 20:01 Phil Dier
2005-09-14 20:55 ` John Stoffel
2005-09-14 21:41 ` Mark Lord
0 siblings, 2 replies; 5+ messages in thread
From: Phil Dier @ 2005-09-14 20:01 UTC (permalink / raw)
To: linux-kernel; +Cc: ziggy, Jack Massari, Scott Holdren
Hi,
I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
I have described before[0]. It still does. Briefly, I have a dual Xeon
2.8 with 4GB ram, 1 adaptec and 1 fusion-mpt scsi device, each connected
to 1 SCSI JBOD with 5 disks apiece[1]. I/O on scsi ids 0 and 1 both
work flawlessly. I built an md device across ids 2 and 3, and the disks
connected to the fusion-mpt are unbearably slow. This behaviour was not
present in 2.6.12.3. When I partition the devices in question and mount
a filesystem on them, they exhibit the same behaviour individually.
[0]: http://www.ussg.iu.edu/hypermail/linux/kernel/0508.3/1233.html
[1]: http://www.icglink.com/debug-2.6.14-rc1.html
Here is what iostat looks like during what should be heavy I/O (my
tesh.sh script described in [1] is running), showing only the relevant
devices:
$ iostat 1 -k | awk '/(Device|md5|md6|md7)/{print $0;}'
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.62 0.00 2.46 244 198312
md6 0.62 0.00 2.46 228 198312
md7 0.61 0.01 2.45 440 197856
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 25.74 0.00 102.97 0 104
md6 25.74 0.00 102.97 0 104
md7 9.90 0.00 39.60 0 40
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 59.41 0.00 237.62 0 240
md6 60.40 3.96 237.62 4 240
md7 20.79 3.96 79.21 4 80
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 100.99 0.00 403.96 0 408
md6 100.99 0.00 403.96 0 408
md7 72.28 0.00 289.11 0 292
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 16242.57 0.00 64970.30 0 65620
md6 16242.57 0.00 64970.30 0 65620
md7 16230.69 0.00 64922.77 0 65572
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 1014.85 0.00 4059.41 0 4100
md6 1014.85 0.00 4059.41 0 4100
md7 1014.85 0.00 4059.41 0 4100
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 3298.02 0.00 13192.08 0 13324
md6 3298.02 0.00 13192.08 0 13324
md7 3298.02 0.00 13192.08 0 13324
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 2789.11 0.00 11156.44 0 11268
md6 2789.11 0.00 11156.44 0 11268
md7 2789.11 0.00 11156.44 0 11268
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
md5 0.00 0.00 0.00 0 0
md6 0.00 0.00 0.00 0 0
md7 0.00 0.00 0.00 0 0
Thanks for looking. Please cc me on replies.
--
Phil Dier (ICGLink.com -- 615 370-1530 x733)
/* vim:set noai nocindent ts=8 sw=8: */
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
2005-09-14 20:01 Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD Phil Dier
@ 2005-09-14 20:55 ` John Stoffel
2005-09-14 21:41 ` Mark Lord
1 sibling, 0 replies; 5+ messages in thread
From: John Stoffel @ 2005-09-14 20:55 UTC (permalink / raw)
To: Phil Dier; +Cc: linux-kernel, ziggy, Jack Massari, Scott Holdren
Phil,
What happens if you pull 1gb or 2gb of memory from your box? Running
a news feed shouldn't be memory intensive and maybe you've run into a
low-mem vs. high-mem issue. It would be an interesting test in any
case.
John
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
2005-09-14 20:01 Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD Phil Dier
2005-09-14 20:55 ` John Stoffel
@ 2005-09-14 21:41 ` Mark Lord
2005-09-14 21:58 ` Phil Dier
2005-09-14 22:08 ` Phil Dier
1 sibling, 2 replies; 5+ messages in thread
From: Mark Lord @ 2005-09-14 21:41 UTC (permalink / raw)
To: Phil Dier; +Cc: linux-kernel, ziggy, Jack Massari, Scott Holdren
Phil Dier wrote:
> Hi,
>
> I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
> I have described before[0]. It still does. Briefly, I have a dual Xeon
..
Do you still have HZ set to 1000 in your .config file? (as per 2.6.12)
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
2005-09-14 21:41 ` Mark Lord
@ 2005-09-14 21:58 ` Phil Dier
2005-09-14 22:08 ` Phil Dier
1 sibling, 0 replies; 5+ messages in thread
From: Phil Dier @ 2005-09-14 21:58 UTC (permalink / raw)
To: linux-kernel; +Cc: ziggy, jack, scott
On Wed, 14 Sep 2005 17:41:58 -0400
Mark Lord <lkml@rtr.ca> wrote:
> Phil Dier wrote:
> > Hi,
> >
> > I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
> > I have described before[0]. It still does. Briefly, I have a dual Xeon
> ..
>
> Do you still have HZ set to 1000 in your .config file? (as per 2.6.12)
No, the config for .14 is using the default of 100. I'm recompiling with
HZ=1000 to see if that makes a difference.
--
Phil Dier (ICGLink.com -- 615 370-1530 x733)
/* vim:set noai nocindent ts=8 sw=8: */
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD
2005-09-14 21:41 ` Mark Lord
2005-09-14 21:58 ` Phil Dier
@ 2005-09-14 22:08 ` Phil Dier
1 sibling, 0 replies; 5+ messages in thread
From: Phil Dier @ 2005-09-14 22:08 UTC (permalink / raw)
To: linux-kernel; +Cc: ziggy, jack, scott
On Wed, 14 Sep 2005 17:41:58 -0400
Mark Lord <lkml@rtr.ca> wrote:
> Phil Dier wrote:
> > Hi,
> >
> > I just tried the 2.6.14-rc1 kernel to see if it exhibits the behaviour
> > I have described before[0]. It still does. Briefly, I have a dual Xeon
> ..
>
> Do you still have HZ set to 1000 in your .config file? (as per 2.6.12)
With HZ at 1000, the speeds seem to be fine. Why would this occur with
just this one controller? Just driver differences?
--
Phil Dier (ICGLink.com -- 615 370-1530 x733)
/* vim:set noai nocindent ts=8 sw=8: */
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2005-09-14 22:08 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2005-09-14 20:01 Slow I/O with SMP, Fusion-MPT and u160 SCSI JBOD Phil Dier
2005-09-14 20:55 ` John Stoffel
2005-09-14 21:41 ` Mark Lord
2005-09-14 21:58 ` Phil Dier
2005-09-14 22:08 ` Phil Dier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox