Re: Degraded I/O performance, since 2.5.41

Re: Degraded I/O performance, since 2.5.41

[Dave Hansen]

Linus Torvalds wrote:
 > In article <3DA4D34F.3070106@us.ibm.com> you write:
 >
 >>When I run a certain large webserver benchmark, I prefer to warm the
 >>pagecache up with the file set first, to cheat a little :)  I grep
 >>through 20 different 500MB file sets in parallel to do this.  It is 
 >>a _lot_ slower in the BK snapshot than in plain 2.5.41.
 >
 > Can you pinpont where this started happening? There aren't that many
 > changes since 2.5.41 that I'd consider suspicious, and it would 
help > to try to pinpoint it a bit.

Using stupid brute force, I narrowed it down to 12 patches.  The only
one that touches generic SCSI code is the queue length one.
1.704.1.2, in my tree from dledford@redhat.com.  Backing it out fixed
my problem.  CC'ing him...

I would guess that it's just making stupid, er.... suboptimal,
decisions about the length for my hardware.  With a controller as
"smart" as the ServeRAID, I would imagine the best approach is to just
throw as many requests at it at one time as you can, then let the
controller sort it out.  Doug, I noticed that you changed a bunch of
code to the aic7xxx_old driver, is this just a lack of optimization in
the ServeRAID driver?

-- 
Dave Hansen
haveblue@us.ibm.com

Re: Degraded I/O performance, since 2.5.41

[James Bottomley]

haveblue@us.ibm.com said:
> I would guess that it's just making stupid, er.... suboptimal,
> decisions about the length for my hardware.  With a controller as
> "smart" as the ServeRAID, I would imagine the best approach is to just
> throw as many requests at it at one time as you can, then let the
> controller sort it out.  Doug, I noticed that you changed a bunch of
> code to the aic7xxx_old driver, is this just a lack of optimization in
> the ServeRAID driver?

OK, I bet this is because the ips driver now no longer actually does tagged 
command queueing.  I'll go an check.

James

Re: Degraded I/O performance, since 2.5.41

[James Bottomley]

[-- Attachment #1: Type: text/plain, Size: 101 bytes --]

OK, this patch should fix it.  Do your performance numbers for ips improve 
again with this?

James


[-- Attachment #2: tmp.diff --]
[-- Type: text/plain , Size: 687 bytes --]

===== drivers/scsi/scsi_scan.c 1.24 vs edited =====
--- 1.24/drivers/scsi/scsi_scan.c	Tue Oct  8 15:45:57 2002
+++ edited/drivers/scsi/scsi_scan.c	Thu Oct 10 17:40:53 2002
@@ -1477,11 +1477,14 @@
 		if (sdt->detect)
 			sdev->attached += (*sdt->detect) (sdev);
 
-	if (sdev->host->hostt->slave_attach != NULL)
+	if (sdev->host->hostt->slave_attach != NULL) {
 		if (sdev->host->hostt->slave_attach(sdev) != 0) {
 			printk(KERN_INFO "scsi_add_lun: failed low level driver attach, setting device offline");
 			sdev->online = FALSE;
 		}
+	} else if(sdev->host->cmd_per_lun) {
+		scsi_adjust_queue_depth(sdev, 0, sdev->host->cmd_per_lun);
+	}
 
 	if (sdevnew != NULL)
 		*sdevnew = sdev;

Re: Degraded I/O performance, since 2.5.41

[Doug Ledford]

On Thu, Oct 10, 2002 at 06:09:14PM -0700, James Bottomley wrote:
> OK, this patch should fix it.  Do your performance numbers for ips improve 
> again with this?

Well, it should help.  Most drivers set cmd_per_lun too low to really be 
good here (and well they should because you only want it big enough to 
keep a CD-R or tape streaming, no more, because it's only intended to be 
used on non-tagged devices, disks should have their depth set via 
scsi_adjust_queue_depth() by the low level driver).  This is in fact the 
right thing to do though, so I've added it to my code here as well since 
it's required in order for cmd_per_lun to do the right thing in regards to 
untagged devices.  The real answer is to update the ServeRAID driver to 
the new tagged queue adjustment mechanism I just wrote an intro to and 
sent to linux-scsi.

As a side note, the generic block layer tagged queueing mechanism may not 
be suitable for a lot of SCSI drivers (I know it's not suitable for mine 
at all).  How many of the drivers in the scsi tree today keep a shared tag 
table between all devices on a host and how many of them have separate tag 
tables for each device?  Show of hands anyone?  Justin, what about the 
modern aic7xxx driver?  Gerard?

> James
> 


-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

block TCQ [was: Re: Degraded I/O performance, since 2.5.41]

[Luben Tuikov]

Doug Ledford wrote:
> 
> As a side note, the generic block layer tagged queueing mechanism may not
> be suitable for a lot of SCSI drivers (I know it's not suitable for mine
> at all).  How many of the drivers in the scsi tree today keep a shared tag
> table between all devices on a host and how many of them have separate tag
> tables for each device?  Show of hands anyone?  Justin, what about the
> modern aic7xxx driver?  Gerard?
> 

Ok. SAM-3 specifies that the initiator assigns task tags (4.9.1).
Who is the initiator?

But, the Abort Task and Abort Task Set (and friends) task management
functions are requested by the application client.
Futhermore, 6.2 Abort Task uses an I_T_L_Q nexus, that is, a tagged task.
So, to cancel an untagged task, one would use the 6.3 Abort Task Set,
which would be consistent since there should be only one untagged task
per initiator.

What are the implications and can we model around this?

Checking up on the definitions, 3.1.4 application client and 3.1.93
the initiator is your driver and the application client is the
SCSI core/block layer/sg/whatever.

AFAIR, IDE happened to support some type of tagging and all of a sudden
TCQ was moved up to the block layer, since SCSI has had it all along.

While this is alright, SAM-3 suggests that TCQ _generation_ is performed
by the SCSI initiator port (i.e. the SCSI LLDD) and for valid reasons...
(again those have to do with the underlying transport/interconnect...)

Thus, the only thing which the block layer would need to know is
_if_ TCQ is supported, but genereration of tags should be left to the
SCSI LLDD.

-- 
Luben