Max IO size

Max IO size

[Mukul Kotwani]

Hello!
Couple of other questions..forgot to ask in the
previous email!

1) When we do 512 byte sequential reads, the mid
level( or someone else) seems to be combining the
requests and sending them down as one big 60k+
request. Can this combining of requests be disabled
somehow?
2) If we do 512k IOs, the mid layer seems to be
breaking it up into 64 k chunks. The Scatter gather
size is set to 240, so there is no reason for it to
break it up. Is there something I am missing? Or is
there a tunable I can set to increase the IO size sent
down to the low level driver?
3) What exactly does the max_sectors in the host
structure do?

Sorry for the long list! I would really appreciate any
response!

Thanks
M


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Max IO size

[Jens Axboe]

On Wed, Jul 31 2002, Mukul Kotwani wrote:
> Hello!
> Couple of other questions..forgot to ask in the
> previous email!
> 
> 1) When we do 512 byte sequential reads, the mid
> level( or someone else) seems to be combining the
> requests and sending them down as one big 60k+
> request. Can this combining of requests be disabled
> somehow?

not the mid layer, it's the block layer doing that for you. you can
limit the size of each i/o with the max_sectors part of the scsi host
template.

> 2) If we do 512k IOs, the mid layer seems to be
> breaking it up into 64 k chunks. The Scatter gather
> size is set to 240, so there is no reason for it to
> break it up. Is there something I am missing? Or is
> there a tunable I can set to increase the IO size sent
> down to the low level driver?

again, the mid layer does not do any combining or breaking up of i/o
requests. the block layer will assemble requests according to the rules
set for it, these rules you control with the host template settings.
these are things like max size, max segments, max segment size, etc.

> 3) What exactly does the max_sectors in the host
> structure do?

see above :-)

-- 
Jens Axboe

Re: Max IO size

[Mukul Kotwani]

Thanks for the reply!

All my values are set high. The sg_tablesize value is
set to 249, max_sectors set to 1024, and I dont think
there is any other value which affects IO size as
such. I didnt see anything called max segment size/max
size in the host template. Any other things I am
missing?

Thanks!
Mukul

--- Jens Axboe <axboe@suse.de> wrote:
> On Wed, Jul 31 2002, Mukul Kotwani wrote:
> > Hello!
> > Couple of other questions..forgot to ask in the
> > previous email!
> > 
> > 1) When we do 512 byte sequential reads, the mid
> > level( or someone else) seems to be combining the
> > requests and sending them down as one big 60k+
> > request. Can this combining of requests be
> disabled
> > somehow?
> 
> not the mid layer, it's the block layer doing that
> for you. you can
> limit the size of each i/o with the max_sectors part
> of the scsi host
> template.
> 
> > 2) If we do 512k IOs, the mid layer seems to be
> > breaking it up into 64 k chunks. The Scatter
> gather
> > size is set to 240, so there is no reason for it
> to
> > break it up. Is there something I am missing? Or
> is
> > there a tunable I can set to increase the IO size
> sent
> > down to the low level driver?
> 
> again, the mid layer does not do any combining or
> breaking up of i/o
> requests. the block layer will assemble requests
> according to the rules
> set for it, these rules you control with the host
> template settings.
> these are things like max size, max segments, max
> segment size, etc.
> 
> > 3) What exactly does the max_sectors in the host
> > structure do?
> 
> see above :-)
> 
> -- 
> Jens Axboe
> 
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
http://vger.kernel.org/majordomo-info.html


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Max IO size

[Mukul Kotwani]

In fact, even with a max_sectors size set to 1, I
still get a 1k IO doing 512 byte reads with IOmeter!

Can someone please clarify?

m


--- Mukul Kotwani <mkotwani@yahoo.com> wrote:
> 
> Thanks for the reply!
> 
> All my values are set high. The sg_tablesize value
> is
> set to 249, max_sectors set to 1024, and I dont
> think
> there is any other value which affects IO size as
> such. I didnt see anything called max segment
> size/max
> size in the host template. Any other things I am
> missing?
> 
> Thanks!
> Mukul
> 
> --- Jens Axboe <axboe@suse.de> wrote:
> > On Wed, Jul 31 2002, Mukul Kotwani wrote:
> > > Hello!
> > > Couple of other questions..forgot to ask in the
> > > previous email!
> > > 
> > > 1) When we do 512 byte sequential reads, the mid
> > > level( or someone else) seems to be combining
> the
> > > requests and sending them down as one big 60k+
> > > request. Can this combining of requests be
> > disabled
> > > somehow?
> > 
> > not the mid layer, it's the block layer doing that
> > for you. you can
> > limit the size of each i/o with the max_sectors
> part
> > of the scsi host
> > template.
> > 
> > > 2) If we do 512k IOs, the mid layer seems to be
> > > breaking it up into 64 k chunks. The Scatter
> > gather
> > > size is set to 240, so there is no reason for it
> > to
> > > break it up. Is there something I am missing? Or
> > is
> > > there a tunable I can set to increase the IO
> size
> > sent
> > > down to the low level driver?
> > 
> > again, the mid layer does not do any combining or
> > breaking up of i/o
> > requests. the block layer will assemble requests
> > according to the rules
> > set for it, these rules you control with the host
> > template settings.
> > these are things like max size, max segments, max
> > segment size, etc.
> > 
> > > 3) What exactly does the max_sectors in the host
> > > structure do?
> > 
> > see above :-)
> > 
> > -- 
> > Jens Axboe
> > 
> > -
> > To unsubscribe from this list: send the line
> > "unsubscribe linux-scsi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Health - Feel better, live better
> http://health.yahoo.com
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
http://vger.kernel.org/majordomo-info.html


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Max IO size

[Jens Axboe]

On Thu, Aug 01 2002, Mukul Kotwani wrote:
> 
> In fact, even with a max_sectors size set to 1, I
> still get a 1k IO doing 512 byte reads with IOmeter!
> 
> Can someone please clarify?

The lowest limit you can set is 8 sectors. Anything below that is not
guarenteed to be honored (it depends on the soft block size of the file
system, or 1kb if there is no fs mounted on the device).

-- 
Jens Axboe

Re: Max IO size

[Jens Axboe]

On Thu, Aug 01 2002, Mukul Kotwani wrote:
> 
> Thanks for the reply!
> 
> All my values are set high. The sg_tablesize value is
> set to 249, max_sectors set to 1024, and I dont think
> there is any other value which affects IO size as
> such. I didnt see anything called max segment size/max
> size in the host template. Any other things I am
> missing?

The max segment size is a property of the queue associated with the SCSI
devices, it's in fact not directly controllable by the low level driver
in the 2.4 kernels. What you can set in this regard is 'use_clustering'.
With that set, the mid layer will attempt to merge contigious pages into
the same scatter gather segment. This usually only provides a benefit
with 1kb transfers (since they will typically be submitted in multiples
anyway from the same page, thus it's easy to cluster them), with eg 4kb
transfer sizes the amount of clustering that will happen is neglible.

-- 
Jens Axboe

Re: Max IO size

[Mukul Kotwani]

Hello!
Thanks again!

If that is the case, then with my settings, is there
any other reason 512k requests are being broken down
into multiple 64k ones? 

Thanks!
Mukul

--- Jens Axboe <axboe@suse.de> wrote:
> On Thu, Aug 01 2002, Mukul Kotwani wrote:
> > 
> > Thanks for the reply!
> > 
> > All my values are set high. The sg_tablesize value
> is
> > set to 249, max_sectors set to 1024, and I dont
> think
> > there is any other value which affects IO size as
> > such. I didnt see anything called max segment
> size/max
> > size in the host template. Any other things I am
> > missing?
> 
> The max segment size is a property of the queue
> associated with the SCSI
> devices, it's in fact not directly controllable by
> the low level driver
> in the 2.4 kernels. What you can set in this regard
> is 'use_clustering'.
> With that set, the mid layer will attempt to merge
> contigious pages into
> the same scatter gather segment. This usually only
> provides a benefit
> with 1kb transfers (since they will typically be
> submitted in multiples
> anyway from the same page, thus it's easy to cluster
> them), with eg 4kb
> transfer sizes the amount of clustering that will
> happen is neglible.
> 
> -- 
> Jens Axboe
> 
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
http://vger.kernel.org/majordomo-info.html


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Performance

[Mukul Kotwani]

Hello!
Does anyone have any performace numbers of a Qlogic
HBA? No of IOps/sec Max MBs /sec?

Thanks!
M


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Performance

[Fabien Salvi]

Mukul Kotwani wrote:
> 
> Hello!
> Does anyone have any performace numbers of a Qlogic
> HBA? No of IOps/sec Max MBs /sec?

I think it depends more on your controllers than on the HBA...

In our tests with a CMD 7240 controller, we have approx. 40 MB/s for
large file transfers.
I don't remember IO/s values...

-------------
Fabien SALVI      Centre de Ressources Informatiques
                  Archamps, France -- http://www.cri74.org
                  PingOO GNU/linux distribution : http://www.pingoo.org

Re: Performance

[Craig Tierney]

For the Qlogic 2200F I am able to push 100 MB/s for
reads and writes using multiple threads to a filesystem
that is striped across mutliple host ports on the SAN.  Backend RAID is a 
Data Direct Networks SAN.  For a single thread to a ext3 filesystem
and 2.4.18, I get about 70 MB/s for reads and writes.

I have no Iops/sec numbers.

Craig


> Hello!
> Does anyone have any performace numbers of a Qlogic
> HBA? No of IOps/sec Max MBs /sec?
> 
> Thanks!
> M
> 
> 
> __________________________________________________
> Do You Yahoo!?
> Yahoo! Health - Feel better, live better
> http://health.yahoo.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Craig Tierney (ctierney@hpti.com)

Re: Performance

[Mukul Kotwani]

Running IOMeter against the Qlogic with IBM 15kRPM
JBODS, Im getting:

1) For 512 byte 100%random reads, I get *just* 260 
IOPs per sec.On Windows with IOMeter, I get about 7000
foir the same config.


2) For 512k 100%sequential reads, I see 30MB per sec.
Windows gives 85MB/sec

I see the similar performance with my driver on Linux.
I dont know whyit is so low, cannot be that low as
compared to windws, a few MB/sec or a few hundred IOPs
per sec difference is OK, but this difference is TOO
huge. I tried a bunch of diff host template params,
but doesnt seem to make a difference.

Has anyone dont any testing with IOMeter? Or is there
any other toold I can use to test IOPs/througput?


Thanks,
Mukul

--- Craig Tierney <ctierney@hpti.com> wrote:
> For the Qlogic 2200F I am able to push 100 MB/s for
> reads and writes using multiple threads to a
> filesystem
> that is striped across mutliple host ports on the
> SAN.  Backend RAID is a 
> Data Direct Networks SAN.  For a single thread to a
> ext3 filesystem
> and 2.4.18, I get about 70 MB/s for reads and
> writes.
> 
> I have no Iops/sec numbers.
> 
> Craig
> 
> 
> > Hello!
> > Does anyone have any performace numbers of a
> Qlogic
> > HBA? No of IOps/sec Max MBs /sec?
> > 
> > Thanks!
> > M
> > 
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Yahoo! Health - Feel better, live better
> > http://health.yahoo.com
> > -
> > To unsubscribe from this list: send the line
> "unsubscribe linux-scsi" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> 
> -- 
> Craig Tierney (ctierney@hpti.com)
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
http://vger.kernel.org/majordomo-info.html


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Performance

[Randy.Dunlap]

On Fri, 2 Aug 2002, Mukul Kotwani wrote:

| Running IOMeter against the Qlogic with IBM 15kRPM
| JBODS, Im getting:
|
| 1) For 512 byte 100%random reads, I get *just* 260
| IOPs per sec.On Windows with IOMeter, I get about 7000
| foir the same config.
|
| 2) For 512k 100%sequential reads, I see 30MB per sec.
| Windows gives 85MB/sec
|
| I see the similar performance with my driver on Linux.
| I dont know whyit is so low, cannot be that low as
| compared to windws, a few MB/sec or a few hundred IOPs
| per sec difference is OK, but this difference is TOO
| huge. I tried a bunch of diff host template params,
| but doesnt seem to make a difference.
|
| Has anyone dont any testing with IOMeter? Or is there
| any other toold I can use to test IOPs/througput?

Hi,

I'm not familiar with people using iometer with Linux.
How does someone do that?
I thought that it was a Windows client app.

A few people do use pgmeter (an iometer clean-room
replacement) on Linux.  SGI and IBM presented a paper
at USENIX just a few weeks ago in which pgmeter was used.
(pgmeter.sf.net).

Also, iozone can measure IOs/second or throughput.
www.iozone.org

~Randy

| Thanks,
| Mukul
|
| --- Craig Tierney <ctierney@hpti.com> wrote:
| > For the Qlogic 2200F I am able to push 100 MB/s for
| > reads and writes using multiple threads to a
| > filesystem
| > that is striped across mutliple host ports on the
| > SAN.  Backend RAID is a
| > Data Direct Networks SAN.  For a single thread to a
| > ext3 filesystem
| > and 2.4.18, I get about 70 MB/s for reads and
| > writes.
| >
| > I have no Iops/sec numbers.
| >
| > Craig
| >
| >
| > > Hello!
| > > Does anyone have any performace numbers of a
| > Qlogic
| > > HBA? No of IOps/sec Max MBs /sec?
| > >
| > > Thanks!
| > > M
| > > __________________________________________________
| > --
| > Craig Tierney (ctierney@hpti.com)
| > -

-- 
~Randy

Re: Performance

[Mukul Kotwani]

Where is PgMeter available? I dont see a binary or
source for it anywhere! Can you please point me to
where it is avilable?

IoMeter source is available on sourceforge.net. I
compiled the dynamo for Linux from there.You can
spcify the IP of the machine which will run the GUI as
a command line param to the dynamo as in: 

dynamo IPaddressOfMachineRunningIoMeter

Run the dynamo on the Linux machine as specified
above, the GUI(Iometer.exe itself!)  on the Windows
machine, and they should establish a connection. Once
the GUI and the dynamo are up and connected, you will
see disks on your Linux machine on the GUI running on
Windows, and can run tests as you normally do!

--- "Randy.Dunlap" <rddunlap@osdl.org> wrote:
> On Fri, 2 Aug 2002, Mukul Kotwani wrote:
> 
> | Running IOMeter against the Qlogic with IBM 15kRPM
> | JBODS, Im getting:
> |
> | 1) For 512 byte 100%random reads, I get *just* 260
> | IOPs per sec.On Windows with IOMeter, I get about
> 7000
> | foir the same config.
> |
> | 2) For 512k 100%sequential reads, I see 30MB per
> sec.
> | Windows gives 85MB/sec
> |
> | I see the similar performance with my driver on
> Linux.
> | I dont know whyit is so low, cannot be that low as
> | compared to windws, a few MB/sec or a few hundred
> IOPs
> | per sec difference is OK, but this difference is
> TOO
> | huge. I tried a bunch of diff host template
> params,
> | but doesnt seem to make a difference.
> |
> | Has anyone dont any testing with IOMeter? Or is
> there
> | any other toold I can use to test IOPs/througput?
> 
> Hi,
> 
> I'm not familiar with people using iometer with
> Linux.
> How does someone do that?
> I thought that it was a Windows client app.
> 
> A few people do use pgmeter (an iometer clean-room
> replacement) on Linux.  SGI and IBM presented a
> paper
> at USENIX just a few weeks ago in which pgmeter was
> used.
> (pgmeter.sf.net).
> 
> Also, iozone can measure IOs/second or throughput.
> www.iozone.org
> 
> ~Randy
> 
> | Thanks,
> | Mukul
> |
> | --- Craig Tierney <ctierney@hpti.com> wrote:
> | > For the Qlogic 2200F I am able to push 100 MB/s
> for
> | > reads and writes using multiple threads to a
> | > filesystem
> | > that is striped across mutliple host ports on
> the
> | > SAN.  Backend RAID is a
> | > Data Direct Networks SAN.  For a single thread
> to a
> | > ext3 filesystem
> | > and 2.4.18, I get about 70 MB/s for reads and
> | > writes.
> | >
> | > I have no Iops/sec numbers.
> | >
> | > Craig
> | >
> | >
> | > > Hello!
> | > > Does anyone have any performace numbers of a
> | > Qlogic
> | > > HBA? No of IOps/sec Max MBs /sec?
> | > >
> | > > Thanks!
> | > > M
> | > >
> __________________________________________________
> | > --
> | > Craig Tierney (ctierney@hpti.com)
> | > -
> 
> -- 
> ~Randy
> 


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Performance

[Randy.Dunlap]

On Sat, 3 Aug 2002, Mukul Kotwani wrote:

| Where is PgMeter available? I dont see a binary or
| source for it anywhere! Can you please point me to
| where it is avilable?

Source code is available only in CVS at
sourceforge.net/projects/pgmeter .
Use cvs to get it, or browse the CVS repository
and download each file.
No binaries available.

| IoMeter source is available on sourceforge.net. I
| compiled the dynamo for Linux from there.You can
| spcify the IP of the machine which will run the GUI as
| a command line param to the dynamo as in:
|
| dynamo IPaddressOfMachineRunningIoMeter
|
| Run the dynamo on the Linux machine as specified
| above, the GUI(Iometer.exe itself!)  on the Windows
| machine, and they should establish a connection. Once
| the GUI and the dynamo are up and connected, you will
| see disks on your Linux machine on the GUI running on
| Windows, and can run tests as you normally do!

Thanks.

~Randy

| --- "Randy.Dunlap" <rddunlap@osdl.org> wrote:
| > On Fri, 2 Aug 2002, Mukul Kotwani wrote:
| >
| > | Running IOMeter against the Qlogic with IBM 15kRPM
| > | JBODS, Im getting:
| > |
| > | 1) For 512 byte 100%random reads, I get *just* 260
| > | IOPs per sec.On Windows with IOMeter, I get about
| > 7000
| > | foir the same config.
| > |
| > | 2) For 512k 100%sequential reads, I see 30MB per
| > sec.
| > | Windows gives 85MB/sec
| > |
| > | I see the similar performance with my driver on
| > Linux.
| > | I dont know whyit is so low, cannot be that low as
| > | compared to windws, a few MB/sec or a few hundred
| > IOPs
| > | per sec difference is OK, but this difference is
| > TOO
| > | huge. I tried a bunch of diff host template
| > params,
| > | but doesnt seem to make a difference.
| > |
| > | Has anyone dont any testing with IOMeter? Or is
| > there
| > | any other toold I can use to test IOPs/througput?
| >
| > Hi,
| >
| > I'm not familiar with people using iometer with
| > Linux.
| > How does someone do that?
| > I thought that it was a Windows client app.
| >
| > A few people do use pgmeter (an iometer clean-room
| > replacement) on Linux.  SGI and IBM presented a
| > paper
| > at USENIX just a few weeks ago in which pgmeter was
| > used.
| > (pgmeter.sf.net).
| >
| > Also, iozone can measure IOs/second or throughput.
| > www.iozone.org
| >
| > ~Randy
| >
| > | Thanks,
| > | Mukul
| > |
| > | --- Craig Tierney <ctierney@hpti.com> wrote:
| > | > For the Qlogic 2200F I am able to push 100 MB/s
| > for
| > | > reads and writes using multiple threads to a
| > | > filesystem
| > | > that is striped across mutliple host ports on
| > the
| > | > SAN.  Backend RAID is a
| > | > Data Direct Networks SAN.  For a single thread
| > to a
| > | > ext3 filesystem
| > | > and 2.4.18, I get about 70 MB/s for reads and
| > | > writes.
| > | >
| > | > I have no Iops/sec numbers.
| > | >
| > | > Craig
| > | >
| > | >
| > | > > Hello!
| > | > > Does anyone have any performace numbers of a
| > | > Qlogic
| > | > > HBA? No of IOps/sec Max MBs /sec?
| > | > >
| > | > > Thanks!
| > | > > M
| > | > >
| > __________________________________________________
| > | > --
| > | > Craig Tierney (ctierney@hpti.com)
| > | > -
| >
| > --
| > ~Randy

Re: Performance

[Simon Trimmer]

Those numbers do seem rather poor, using Matt Jacob's linux qlogic driver as
an initiator I can easily get 18,000 ops/s on 230x series isp cards with a
userspace app. I don't have recent figures for isp 220x cards but around 7k
ops/s matches some old notes and given the right workload you can pretty much
max out the channel.

Most of the block io experts have steered clear of the thread, this might be
because a lot of performance measurement depends on the load generator, your
scsi targets, the workload actually presented to them, whether OS readahead /
request merging fires etc etc etc. It's not so clear cut and simple anymore
for a straight answer!

Chances are the qlogic isp cards have more than enough grunt for most people.
If not, they are probably doing something wrong or can afford multiport
variants! :)

-Simon
Simon Trimmer <simon@urbanmyth.org>


On Fri, 2 Aug 2002, Mukul Kotwani wrote:
> Running IOMeter against the Qlogic with IBM 15kRPM
> JBODS, Im getting:
>
> 1) For 512 byte 100%random reads, I get *just* 260
> IOPs per sec.On Windows with IOMeter, I get about 7000
> foir the same config.
>
>
> 2) For 512k 100%sequential reads, I see 30MB per sec.
> Windows gives 85MB/sec
>
> I see the similar performance with my driver on Linux.
> I dont know whyit is so low, cannot be that low as
> compared to windws, a few MB/sec or a few hundred IOPs
> per sec difference is OK, but this difference is TOO
> huge. I tried a bunch of diff host template params,
> but doesnt seem to make a difference.
>
> Has anyone dont any testing with IOMeter? Or is there
> any other toold I can use to test IOPs/througput?
>
>
> Thanks,
> Mukul
>
> --- Craig Tierney <ctierney@hpti.com> wrote:
> > For the Qlogic 2200F I am able to push 100 MB/s for
> > reads and writes using multiple threads to a
> > filesystem
> > that is striped across mutliple host ports on the
> > SAN.  Backend RAID is a
> > Data Direct Networks SAN.  For a single thread to a
> > ext3 filesystem
> > and 2.4.18, I get about 70 MB/s for reads and
> > writes.
> >
> > I have no Iops/sec numbers.
> >
> > Craig
> >
> >
> > > Hello!
> > > Does anyone have any performace numbers of a Qlogic
> > > HBA? No of IOps/sec Max MBs /sec?
> > >
> > > Thanks!
> > > M

Re: Performance

[Mukul Kotwani]

I do get about 23000 IOPs if I do sequential 512 byte
reads. And there, the block driver seems to be merging
requests into one big chunk. Thats the reason I tried
random reads instead of sequential , and I have the
QLA2340, which I guess is better than the 220x series.
And the sequential performance is not the correct
indicator because of the merging. Thats the reason I
tried random. Did u get 18k IOPS on random or
sequential? And I am using the latest JBODs from IBM,
15k RPM drives, so drives is not the reason got poor
performance.

The load generator is IOMeter/dynamo,which is pretty
much the standard in the Windows world. And the same
card with the same storage using Qlogic drivers gives
7000 IOPs on random reads to 13 LUns with 20
outstanding IOs on Windows, where Linux with the same
settings gives 250! And I did try to vary the
queuedepth of the drivers, and that does not seem to
make a difference. So its a straight comparison, same
card, same storage, same load generator.

Same for the 512k sequential reads. I see 64k requests
on the wire even though I claim to support 1024
sectors  and a scatter gather list of 248! 

Any pointers?

Thanks,
Mukul

 
--- Simon Trimmer <simon@urbanmyth.org> wrote:
> Those numbers do seem rather poor, using Matt
> Jacob's linux qlogic driver as
> an initiator I can easily get 18,000 ops/s on 230x
> series isp cards with a
> userspace app. I don't have recent figures for isp
> 220x cards but around 7k
> ops/s matches some old notes and given the right
> workload you can pretty much
> max out the channel.
> 
> Most of the block io experts have steered clear of
> the thread, this might be
> because a lot of performance measurement depends on
> the load generator, your
> scsi targets, the workload actually presented to
> them, whether OS readahead /
> request merging fires etc etc etc. It's not so clear
> cut and simple anymore
> for a straight answer!
> 
> Chances are the qlogic isp cards have more than
> enough grunt for most people.
> If not, they are probably doing something wrong or
> can afford multiport
> variants! :)
> 
> -Simon
> Simon Trimmer <simon@urbanmyth.org>
> 
> 
> On Fri, 2 Aug 2002, Mukul Kotwani wrote:
> > Running IOMeter against the Qlogic with IBM 15kRPM
> > JBODS, Im getting:
> >
> > 1) For 512 byte 100%random reads, I get *just* 260
> > IOPs per sec.On Windows with IOMeter, I get about
> 7000
> > foir the same config.
> >
> >
> > 2) For 512k 100%sequential reads, I see 30MB per
> sec.
> > Windows gives 85MB/sec
> >
> > I see the similar performance with my driver on
> Linux.
> > I dont know whyit is so low, cannot be that low as
> > compared to windws, a few MB/sec or a few hundred
> IOPs
> > per sec difference is OK, but this difference is
> TOO
> > huge. I tried a bunch of diff host template
> params,
> > but doesnt seem to make a difference.
> >
> > Has anyone dont any testing with IOMeter? Or is
> there
> > any other toold I can use to test IOPs/througput?
> >
> >
> > Thanks,
> > Mukul
> >
> > --- Craig Tierney <ctierney@hpti.com> wrote:
> > > For the Qlogic 2200F I am able to push 100 MB/s
> for
> > > reads and writes using multiple threads to a
> > > filesystem
> > > that is striped across mutliple host ports on
> the
> > > SAN.  Backend RAID is a
> > > Data Direct Networks SAN.  For a single thread
> to a
> > > ext3 filesystem
> > > and 2.4.18, I get about 70 MB/s for reads and
> > > writes.
> > >
> > > I have no Iops/sec numbers.
> > >
> > > Craig
> > >
> > >
> > > > Hello!
> > > > Does anyone have any performace numbers of a
> Qlogic
> > > > HBA? No of IOps/sec Max MBs /sec?
> > > >
> > > > Thanks!
> > > > M
> 
> 
> 


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Performance

[Jeremy Higdon]

We've seen about 15000 IOPS from a 2200 and nearly 40000 from a 2310.
With the proper test and proper hardware, you ought to see up to 102 MB/s
on disk reads with a 2200 and 204 MB/s with the 2300 (10^6 MB).

However, you may have trouble matching this in the Linux block layer.
Perhaps if you try the sg driver with direct I/O . . . .

I question the 100% random results that you got from Windows.  100%
random implies no cache hits, which would leave you with the raw
IOPS that the drive can supply (260 sounds in the right ballpark).

It would be impossible for the drive to supply 7000 IOPS in a
random workload.  If you were supplying out of the drive cache or
Windows cache, then 7000 seems more reasonable (though perhaps not
a very interesting number).

jeremy

Re: Performance

[Mukul Kotwani]

Thanks for the reply Jeremy!

Can you point me to the proper test which I can use to
test it? I think I have pretty good servers and
storage. What did u guys use..and was there any tuning
of the OS required?

For the IOPs, I guess it must have been the Windows
cache then, because ths storage used was the same in
both cases.

Mukul


--- Jeremy Higdon <jeremy@classic.engr.sgi.com> wrote:
> We've seen about 15000 IOPS from a 2200 and nearly
> 40000 from a 2310.
> With the proper test and proper hardware, you ought
> to see up to 102 MB/s
> on disk reads with a 2200 and 204 MB/s with the 2300
> (10^6 MB).
> 
> However, you may have trouble matching this in the
> Linux block layer.
> Perhaps if you try the sg driver with direct I/O . .
> . .
> 
> I question the 100% random results that you got from
> Windows.  100%
> random implies no cache hits, which would leave you
> with the raw
> IOPS that the drive can supply (260 sounds in the
> right ballpark).
> 
> It would be impossible for the drive to supply 7000
> IOPS in a
> random workload.  If you were supplying out of the
> drive cache or
> Windows cache, then 7000 seems more reasonable
> (though perhaps not
> a very interesting number).
> 
> jeremy
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
http://vger.kernel.org/majordomo-info.html


__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

Re: Performance

[Jeremy Higdon]

On Aug 4, 12:39am, Mukul Kotwani wrote:
> 
> Thanks for the reply Jeremy!
> 
> Can you point me to the proper test which I can use to
> test it? I think I have pretty good servers and
> storage. What did u guys use..and was there any tuning
> of the OS required?
> 
> For the IOPs, I guess it must have been the Windows
> cache then, because ths storage used was the same in
> both cases.
> 
> Mukul


Well, a later post from you indicated you were using 13 luns
(I presume that means 13 of these 15K drives).  That would
be 538 IOPS per drive, which sounds a little high (2ms per
I/O would account for 1/2 rotation and no seek/settle time).

The tests we've run were not using stock Linux SCSI, so it
might be hard for you to duplicate.  Have you tried using
the raw driver interface to sd (/dev/raw), or the sg
benchmark tools?  I believe the raw interface should give
your the IOPS and the sg interface would give you the MB/s
(and also the IOPS perhaps).

A couple of years ago, we had some patches to the raw interface
and block layer that allowed kiobufs to be passed directly to
the SCSI interface.  That was nice because we avoided the
CPU overhead of deconstruction and reconstruction of large
I/O requests.  However, it was messy.  I believe that the
2.5 changes with bio should help a lot in this area, without
the mess.

jeremy