Scsi midlayer question

Scsi midlayer question

[Frank Borich]

I can see from a fibre channel trace that my 1 MB I/O transfers are
getting cut up into little chunks.
Is there any options that can be set when loading the scsi mid layer as
a module that will improve
io performance or prevent this breakdown for the sg,sd, or even raw
drivers.  I am not a driver developer 
so please excuse my ignorance here.   Any info or points in the right
direction would be greatly apprecited.
I am using the 2.4.20 kernel.  
 
Thanks.
 
Franco Borich

Frank Borich
------------------------------------------------------------------
Xyratex Inc.
Systems Engineer
Direct 630 654 6089
Fax 630 325 2513
e-mail frank_borich@us.xyratex.com
 

Re: Scsi midlayer question

[Arjan van de Ven]

[-- Attachment #1.1: Type: text/plain, Size: 558 bytes --]

On Fri, 2004-07-16 at 19:55, Frank Borich wrote:
> I can see from a fibre channel trace that my 1 MB I/O transfers are
> getting cut up into little chunks.
> Is there any options that can be set when loading the scsi mid layer as
> a module that will improve
> io performance or prevent this breakdown for the sg,sd, or even raw
> drivers.  I am not a driver developer 
> so please excuse my ignorance here.   Any info or points in the right
> direction would be greatly apprecited.
> I am using the 2.4.20 kernel.  

try the patch I attached.


[-- Attachment #1.2: w.patch --]
[-- Type: text/x-patch, Size: 604 bytes --]

--- linux/mm/page_alloc.c.org	2004-06-24 17:36:22.042460920 +0200
+++ linux/mm/page_alloc.c	2004-06-24 17:38:12.361689848 +0200
@@ -282,15 +282,13 @@
 	unsigned long size = 1 << high;
 
 	while (high > low) {
-		if (BAD_RANGE(zone,page))
-			BUG();
 		area--;
 		high--;
 		size >>= 1;
-		list_add(&(page)->list, &(area)->free_list);
-		MARK_USED(index, high, area);
-		index += size;
-		page += size;
+		if (BAD_RANGE(zone,&page[size]))
+			BUG();
+		list_add(&page[size].list, &(area)->free_list);
+		MARK_USED(index + size, high, area);
 	}
 	if (BAD_RANGE(zone,page))
 		BUG();

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Scsi midlayer question

[Bryan Henderson]

>I can see from a fibre channel trace that my 1 MB I/O transfers are
>getting cut up into little chunks.

What is "my 1 MB I/O transfer"?  How are you generating or seeing this 
transfer?  Sg device?

I ask because sometimes when I hear this question, the transfer is a 
transfer to the page cache, so it has only a loose association with 
transfers on the fibre channel.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

RE: Scsi midlayer question

[Frank Borich]

I am generating this transfer using a c program using read/write system
calls with the open flag
O_WRONLY to sd driver.

Frank Borich
------------------------------------------------------------------
Xyratex Inc.
Systems Engineer
Direct 630 654 6089
Fax 630 325 2513
e-mail frank_borich@us.xyratex.com
 
 

>>-----Original Message-----
>>From: Bryan Henderson [mailto:hbryan@us.ibm.com] 
>>Sent: Friday, July 16, 2004 1:25 PM
>>To: Frank Borich
>>Cc: linux-scsi@vger.kernel.org
>>Subject: Re: Scsi midlayer question
>>
>>>I can see from a fibre channel trace that my 1 MB I/O transfers are 
>>>getting cut up into little chunks.
>>
>>What is "my 1 MB I/O transfer"?  How are you generating or 
>>seeing this transfer?  Sg device?
>>
>>I ask because sometimes when I hear this question, the 
>>transfer is a transfer to the page cache, so it has only a 
>>loose association with transfers on the fibre channel.
>>
>>--
>>Bryan Henderson                          IBM Almaden Research Center
>>San Jose CA                              Filesystems
>>
>>

Re: Scsi midlayer question

[Marcelo Tosatti]

On Fri, Jul 16, 2004 at 08:08:04PM +0200, Arjan van de Ven wrote:
> On Fri, 2004-07-16 at 19:55, Frank Borich wrote:
> > I can see from a fibre channel trace that my 1 MB I/O transfers are
> > getting cut up into little chunks.
> > Is there any options that can be set when loading the scsi mid layer as
> > a module that will improve
> > io performance or prevent this breakdown for the sg,sd, or even raw
> > drivers.  I am not a driver developer 
> > so please excuse my ignorance here.   Any info or points in the right
> > direction would be greatly apprecited.
> > I am using the 2.4.20 kernel.  
> 
> try the patch I attached.
> 

> --- linux/mm/page_alloc.c.org	2004-06-24 17:36:22.042460920 +0200
> +++ linux/mm/page_alloc.c	2004-06-24 17:38:12.361689848 +0200
> @@ -282,15 +282,13 @@
>  	unsigned long size = 1 << high;
>  
>  	while (high > low) {
> -		if (BAD_RANGE(zone,page))
> -			BUG();
>  		area--;
>  		high--;
>  		size >>= 1;
> -		list_add(&(page)->list, &(area)->free_list);
> -		MARK_USED(index, high, area);
> -		index += size;
> -		page += size;
> +		if (BAD_RANGE(zone,&page[size]))
> +			BUG();
> +		list_add(&page[size].list, &(area)->free_list);
> +		MARK_USED(index + size, high, area);
>  	}
>  	if (BAD_RANGE(zone,page))
>  		BUG();

Arjan, is this a v2.6 based patch? 

v2.4 current doesnt contain such code you are patching...

Re: Scsi midlayer question

[Arjan van de Ven]

[-- Attachment #1: Type: text/plain, Size: 321 bytes --]

On Mon, Aug 09, 2004 at 07:39:56PM -0300, Marcelo Tosatti wrote:
> >  		BUG();
> 
> Arjan, is this a v2.6 based patch? 

no a 2.4.21-era patch

 
> v2.4 current doesnt contain such code you are patching...

it'll have code quite similar, I mean this code hardly changed since 2.4.0
and 2.6.x until this patch got merged.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Scsi midlayer question

[Marcelo Tosatti]

On Tue, Aug 10, 2004 at 10:05:57AM +0200, Arjan van de Ven wrote:
> On Mon, Aug 09, 2004 at 07:39:56PM -0300, Marcelo Tosatti wrote:
> > >  		BUG();
> > 
> > Arjan, is this a v2.6 based patch? 
> 
> no a 2.4.21-era patch
> 
>  
> > v2.4 current doesnt contain such code you are patching...
> 
> it'll have code quite similar, I mean this code hardly changed since 2.4.0
> and 2.6.x until this patch got merged.

Should have read the code before replying, OK now I see, its wli's discover
to not handle higher order pages in "reverse order" but instead
in "order", yes?

Have you done any tests with it on v2.4 based kernels? 

I'm a bit skeptic about it though.

Frank, can you try this patch?

RE: Scsi midlayer question

[Frank Borich]

Let me see if I can grab some resources to give this patch a try.  I
will get back to you ASAP.

Frank Borich
------------------------------------------------------------------
Xyratex Inc.
Systems Engineer
Direct 630 654 6089
Fax 630 325 2513
e-mail frank_borich@us.xyratex.com
 
 

>>-----Original Message-----
>>From: Marcelo Tosatti [mailto:marcelo.tosatti@cyclades.com] 
>>Sent: Tuesday, August 10, 2004 7:40 AM
>>To: Arjan van de Ven
>>Cc: Frank Borich; linux-scsi@vger.kernel.org
>>Subject: Re: Scsi midlayer question
>>
>>On Tue, Aug 10, 2004 at 10:05:57AM +0200, Arjan van de Ven wrote:
>>> On Mon, Aug 09, 2004 at 07:39:56PM -0300, Marcelo Tosatti wrote:
>>> > >  		BUG();
>>> > 
>>> > Arjan, is this a v2.6 based patch? 
>>> 
>>> no a 2.4.21-era patch
>>> 
>>>  
>>> > v2.4 current doesnt contain such code you are patching...
>>> 
>>> it'll have code quite similar, I mean this code hardly 
>>changed since 
>>> 2.4.0 and 2.6.x until this patch got merged.
>>
>>Should have read the code before replying, OK now I see, its 
>>wli's discover to not handle higher order pages in "reverse 
>>order" but instead in "order", yes?
>>
>>Have you done any tests with it on v2.4 based kernels? 
>>
>>I'm a bit skeptic about it though.
>>
>>Frank, can you try this patch?
>>
>>
>>
>>
>>

RE: Scsi midlayer question

[Frank Borich]

I have someone that will try the patch out on a stock 2.4 kernel
sometime tomorrow.
I will keep you posted.  Something else I noticed on the FC trace was
that some of the
sequential I/O CDBS were out of order.  Would this patch help with this
as well.


Frank Borich
------------------------------------------------------------------
Xyratex Inc.
Systems Engineer
Direct 630 654 6089
Fax 630 325 2513
e-mail frank_borich@us.xyratex.com
 
 

>>-----Original Message-----
>>From: Marcelo Tosatti [mailto:marcelo.tosatti@cyclades.com] 
>>Sent: Tuesday, August 10, 2004 7:40 AM
>>To: Arjan van de Ven
>>Cc: Frank Borich; linux-scsi@vger.kernel.org
>>Subject: Re: Scsi midlayer question
>>
>>On Tue, Aug 10, 2004 at 10:05:57AM +0200, Arjan van de Ven wrote:
>>> On Mon, Aug 09, 2004 at 07:39:56PM -0300, Marcelo Tosatti wrote:
>>> > >  		BUG();
>>> > 
>>> > Arjan, is this a v2.6 based patch? 
>>> 
>>> no a 2.4.21-era patch
>>> 
>>>  
>>> > v2.4 current doesnt contain such code you are patching...
>>> 
>>> it'll have code quite similar, I mean this code hardly 
>>changed since 
>>> 2.4.0 and 2.6.x until this patch got merged.
>>
>>Should have read the code before replying, OK now I see, its 
>>wli's discover to not handle higher order pages in "reverse 
>>order" but instead in "order", yes?
>>
>>Have you done any tests with it on v2.4 based kernels? 
>>
>>I'm a bit skeptic about it though.
>>
>>Frank, can you try this patch?
>>
>>
>>
>>
>>

RE: Scsi midlayer question

[Frank Borich]

I was able to try the patch.  

Without the patch, a single 1 MB transfer (dd if=/dev/zero of=/dev/sdb
bs=1024768 count=1)
gets broken down into 8 * 128 K chunks

With the patch, the same transfer gets broken down into 4 * 256 K chunks

If you want me to try something else let me know.  


Frank Borich
------------------------------------------------------------------
Xyratex Inc.
Systems Engineer
Direct 630 654 6089
Fax 630 325 2513
e-mail frank_borich@us.xyratex.com
 
 

>>-----Original Message-----
>>From: Marcelo Tosatti [mailto:marcelo.tosatti@cyclades.com] 
>>Sent: Tuesday, August 10, 2004 7:40 AM
>>To: Arjan van de Ven
>>Cc: Frank Borich; linux-scsi@vger.kernel.org
>>Subject: Re: Scsi midlayer question
>>
>>On Tue, Aug 10, 2004 at 10:05:57AM +0200, Arjan van de Ven wrote:
>>> On Mon, Aug 09, 2004 at 07:39:56PM -0300, Marcelo Tosatti wrote:
>>> > >  		BUG();
>>> > 
>>> > Arjan, is this a v2.6 based patch? 
>>> 
>>> no a 2.4.21-era patch
>>> 
>>>  
>>> > v2.4 current doesnt contain such code you are patching...
>>> 
>>> it'll have code quite similar, I mean this code hardly 
>>changed since 
>>> 2.4.0 and 2.6.x until this patch got merged.
>>
>>Should have read the code before replying, OK now I see, its 
>>wli's discover to not handle higher order pages in "reverse 
>>order" but instead in "order", yes?
>>
>>Have you done any tests with it on v2.4 based kernels? 
>>
>>I'm a bit skeptic about it though.
>>
>>Frank, can you try this patch?
>>
>>
>>
>>
>>