* [PATCH] SCSI sym53c8xx_2: bigger transfer limits
@ 2006-02-28 19:14 Kai Makisara
2006-02-28 20:41 ` Matthew Wilcox
2006-03-01 15:29 ` Jens Axboe
0 siblings, 2 replies; 14+ messages in thread
From: Kai Makisara @ 2006-02-28 19:14 UTC (permalink / raw)
To: matthew; +Cc: linux-scsi
This patch enables clustering and sets max_sectors to 0xffff to enable
reading and writing of large blocks with tapes (and large transfers with
sg). This change is needed after the sg and st drivers started using
chained bios through scsi_request_async() in 2.6.16.
Signed-off-by: Kai Makisara <kai.makisara@kolumbus.fi>
--- linux-2.6.16-rc5/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-04 13:25:48.000000000 +0200
+++ linux-2.6.16-rc5-k1/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-18 09:45:24.000000000 +0200
@@ -1978,7 +1978,8 @@ static struct scsi_host_template sym2_te
.eh_bus_reset_handler = sym53c8xx_eh_bus_reset_handler,
.eh_host_reset_handler = sym53c8xx_eh_host_reset_handler,
.this_id = 7,
- .use_clustering = DISABLE_CLUSTERING,
+ .use_clustering = ENABLE_CLUSTERING,
+ .max_sectors = 0xFFFF,
#ifdef SYM_LINUX_PROC_INFO_SUPPORT
.proc_info = sym53c8xx_proc_info,
.proc_name = NAME53C8XX,
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-02-28 19:14 [PATCH] SCSI sym53c8xx_2: bigger transfer limits Kai Makisara
@ 2006-02-28 20:41 ` Matthew Wilcox
2006-03-01 15:29 ` Jens Axboe
1 sibling, 0 replies; 14+ messages in thread
From: Matthew Wilcox @ 2006-02-28 20:41 UTC (permalink / raw)
To: Kai Makisara; +Cc: linux-scsi
On Tue, Feb 28, 2006 at 09:14:31PM +0200, Kai Makisara wrote:
> This patch enables clustering and sets max_sectors to 0xffff to enable
> reading and writing of large blocks with tapes (and large transfers with
> sg). This change is needed after the sg and st drivers started using
> chained bios through scsi_request_async() in 2.6.16.
Seems entirely reasonable to me. Did you want this in 2.6.16, or is
2.6.17 soon enough?
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-02-28 19:14 [PATCH] SCSI sym53c8xx_2: bigger transfer limits Kai Makisara
2006-02-28 20:41 ` Matthew Wilcox
@ 2006-03-01 15:29 ` Jens Axboe
2006-03-01 15:39 ` James Bottomley
2006-03-01 20:34 ` Kai Makisara
1 sibling, 2 replies; 14+ messages in thread
From: Jens Axboe @ 2006-03-01 15:29 UTC (permalink / raw)
To: Kai Makisara; +Cc: matthew, linux-scsi
On Tue, Feb 28 2006, Kai Makisara wrote:
> This patch enables clustering and sets max_sectors to 0xffff to enable
> reading and writing of large blocks with tapes (and large transfers with
> sg). This change is needed after the sg and st drivers started using
> chained bios through scsi_request_async() in 2.6.16.
>
> Signed-off-by: Kai Makisara <kai.makisara@kolumbus.fi>
>
> --- linux-2.6.16-rc5/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-04 13:25:48.000000000 +0200
> +++ linux-2.6.16-rc5-k1/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-18 09:45:24.000000000 +0200
> @@ -1978,7 +1978,8 @@ static struct scsi_host_template sym2_te
> .eh_bus_reset_handler = sym53c8xx_eh_bus_reset_handler,
> .eh_host_reset_handler = sym53c8xx_eh_host_reset_handler,
> .this_id = 7,
> - .use_clustering = DISABLE_CLUSTERING,
> + .use_clustering = ENABLE_CLUSTERING,
> + .max_sectors = 0xFFFF,
Strictly speaking, the clustering bit is unrelated. I seem to recall
Gerard years ago talking about some sym chips that did not like
clustering, hence it was disabled.
--
Jens Axboe
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 15:29 ` Jens Axboe
@ 2006-03-01 15:39 ` James Bottomley
2006-03-01 15:54 ` Jens Axboe
2006-03-01 16:00 ` Douglas Gilbert
2006-03-01 20:34 ` Kai Makisara
1 sibling, 2 replies; 14+ messages in thread
From: James Bottomley @ 2006-03-01 15:39 UTC (permalink / raw)
To: Jens Axboe; +Cc: Kai Makisara, matthew, linux-scsi
On Wed, 2006-03-01 at 16:29 +0100, Jens Axboe wrote:
> Strictly speaking, the clustering bit is unrelated. I seem to recall
> Gerard years ago talking about some sym chips that did not like
> clustering, hence it was disabled.
Yes, I remember that too ... I've never been able to find out which
chip, though ... the scripts all seem happily coded for variable size sg
segments.
However, given the new way 2.6 does memory allocations,
ENABLE_CLUSTERING will probably make quite a difference to the size of
the sg list ... since we try to allocate contiguous pages, physical
merging becomes much more of a possibility (I think I last measured it
at around 30% of all SG tables, as opposed to <1% with the old
allocation method).
James
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 15:39 ` James Bottomley
@ 2006-03-01 15:54 ` Jens Axboe
2006-03-01 16:00 ` Douglas Gilbert
1 sibling, 0 replies; 14+ messages in thread
From: Jens Axboe @ 2006-03-01 15:54 UTC (permalink / raw)
To: James Bottomley; +Cc: Kai Makisara, matthew, linux-scsi
On Wed, Mar 01 2006, James Bottomley wrote:
> On Wed, 2006-03-01 at 16:29 +0100, Jens Axboe wrote:
> > Strictly speaking, the clustering bit is unrelated. I seem to recall
> > Gerard years ago talking about some sym chips that did not like
> > clustering, hence it was disabled.
>
> Yes, I remember that too ... I've never been able to find out which
> chip, though ... the scripts all seem happily coded for variable size sg
> segments.
>
> However, given the new way 2.6 does memory allocations,
> ENABLE_CLUSTERING will probably make quite a difference to the size of
> the sg list ... since we try to allocate contiguous pages, physical
> merging becomes much more of a possibility (I think I last measured it
> at around 30% of all SG tables, as opposed to <1% with the old
> allocation method).
So if we want to enable clustering, it should be done as a separate
patch just in case...
--
Jens Axboe
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 15:39 ` James Bottomley
2006-03-01 15:54 ` Jens Axboe
@ 2006-03-01 16:00 ` Douglas Gilbert
2006-03-01 16:11 ` Jens Axboe
2006-03-01 16:17 ` James Bottomley
1 sibling, 2 replies; 14+ messages in thread
From: Douglas Gilbert @ 2006-03-01 16:00 UTC (permalink / raw)
To: James Bottomley; +Cc: Jens Axboe, Kai Makisara, matthew, linux-scsi
James Bottomley wrote:
> On Wed, 2006-03-01 at 16:29 +0100, Jens Axboe wrote:
>
>>Strictly speaking, the clustering bit is unrelated. I seem to recall
>>Gerard years ago talking about some sym chips that did not like
>>clustering, hence it was disabled.
>
>
> Yes, I remember that too ... I've never been able to find out which
> chip, though ... the scripts all seem happily coded for variable size sg
> segments.
>
> However, given the new way 2.6 does memory allocations,
> ENABLE_CLUSTERING will probably make quite a difference to the size of
> the sg list ... since we try to allocate contiguous pages, physical
> merging becomes much more of a possibility (I think I last measured it
> at around 30% of all SG tables, as opposed to <1% with the old
> allocation method).
James,
So the maximum data carrying size of a scatter gather list
is not deterministic? Is the worst case (page_size * SG_ALL)?
Doug Gilbert
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 16:00 ` Douglas Gilbert
@ 2006-03-01 16:11 ` Jens Axboe
2006-03-01 16:17 ` James Bottomley
1 sibling, 0 replies; 14+ messages in thread
From: Jens Axboe @ 2006-03-01 16:11 UTC (permalink / raw)
To: Douglas Gilbert; +Cc: James Bottomley, Kai Makisara, matthew, linux-scsi
On Wed, Mar 01 2006, Douglas Gilbert wrote:
> James Bottomley wrote:
> > On Wed, 2006-03-01 at 16:29 +0100, Jens Axboe wrote:
> >
> >>Strictly speaking, the clustering bit is unrelated. I seem to recall
> >>Gerard years ago talking about some sym chips that did not like
> >>clustering, hence it was disabled.
> >
> >
> > Yes, I remember that too ... I've never been able to find out which
> > chip, though ... the scripts all seem happily coded for variable size sg
> > segments.
> >
> > However, given the new way 2.6 does memory allocations,
> > ENABLE_CLUSTERING will probably make quite a difference to the size of
> > the sg list ... since we try to allocate contiguous pages, physical
> > merging becomes much more of a possibility (I think I last measured it
> > at around 30% of all SG tables, as opposed to <1% with the old
> > allocation method).
>
> James,
> So the maximum data carrying size of a scatter gather list
> is not deterministic? Is the worst case (page_size * SG_ALL)?
It's block layer restricted (well not because of the block layer, but
the limits that the hardware tolds us that it has), typically page_size
* max_sg_entries is the correct answer but it may of course be more.
--
Jens Axboe
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 16:00 ` Douglas Gilbert
2006-03-01 16:11 ` Jens Axboe
@ 2006-03-01 16:17 ` James Bottomley
1 sibling, 0 replies; 14+ messages in thread
From: James Bottomley @ 2006-03-01 16:17 UTC (permalink / raw)
To: dougg; +Cc: Jens Axboe, Kai Makisara, matthew, linux-scsi
On Wed, 2006-03-01 at 11:00 -0500, Douglas Gilbert wrote:
> So the maximum data carrying size of a scatter gather list
> is not deterministic? Is the worst case (page_size * SG_ALL)?
Correct, essentially. The size is bounded by the number of elements,
but there's no fixed size for those elements unless clustering is
disabled, in which case PAGE_SIZE is the fixed size. However, in most
machines fragmentation effects will cause a certain amount of
unpredictability in the true size of the request. And, obviously, for
IOMMU machines, the block layer limits are about the only factor.
However, having users rely on unpredictable effects like physical
merging or the presence of an iommu isn't really best practice, so
practially, the user must assume PAGE_SIZE * SG_ALL (assming the actual
HBA uses SG_ALL, which not all do)
James
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 15:29 ` Jens Axboe
2006-03-01 15:39 ` James Bottomley
@ 2006-03-01 20:34 ` Kai Makisara
2006-03-01 20:43 ` Jens Axboe
1 sibling, 1 reply; 14+ messages in thread
From: Kai Makisara @ 2006-03-01 20:34 UTC (permalink / raw)
To: Jens Axboe; +Cc: matthew, linux-scsi
On Wed, 1 Mar 2006, Jens Axboe wrote:
> On Tue, Feb 28 2006, Kai Makisara wrote:
> > This patch enables clustering and sets max_sectors to 0xffff to enable
> > reading and writing of large blocks with tapes (and large transfers with
> > sg). This change is needed after the sg and st drivers started using
> > chained bios through scsi_request_async() in 2.6.16.
> >
> > Signed-off-by: Kai Makisara <kai.makisara@kolumbus.fi>
> >
> > --- linux-2.6.16-rc5/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-04 13:25:48.000000000 +0200
> > +++ linux-2.6.16-rc5-k1/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-18 09:45:24.000000000 +0200
> > @@ -1978,7 +1978,8 @@ static struct scsi_host_template sym2_te
> > .eh_bus_reset_handler = sym53c8xx_eh_bus_reset_handler,
> > .eh_host_reset_handler = sym53c8xx_eh_host_reset_handler,
> > .this_id = 7,
> > - .use_clustering = DISABLE_CLUSTERING,
> > + .use_clustering = ENABLE_CLUSTERING,
> > + .max_sectors = 0xFFFF,
>
> Strictly speaking, the clustering bit is unrelated.
It is related. The number of scatter/gather segments in the SCSI HBAs is
limited. In order to fit the large requests into these limits, the
adjacent pages (that have been split to bios) have to be recombined.
> I seem to recall
> Gerard years ago talking about some sym chips that did not like
> clustering, hence it was disabled.
>
Facts?
--
Kai
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 20:34 ` Kai Makisara
@ 2006-03-01 20:43 ` Jens Axboe
2006-03-01 22:24 ` Kai Makisara
0 siblings, 1 reply; 14+ messages in thread
From: Jens Axboe @ 2006-03-01 20:43 UTC (permalink / raw)
To: Kai Makisara; +Cc: matthew, linux-scsi
On Wed, Mar 01 2006, Kai Makisara wrote:
> On Wed, 1 Mar 2006, Jens Axboe wrote:
>
> > On Tue, Feb 28 2006, Kai Makisara wrote:
> > > This patch enables clustering and sets max_sectors to 0xffff to enable
> > > reading and writing of large blocks with tapes (and large transfers with
> > > sg). This change is needed after the sg and st drivers started using
> > > chained bios through scsi_request_async() in 2.6.16.
> > >
> > > Signed-off-by: Kai Makisara <kai.makisara@kolumbus.fi>
> > >
> > > --- linux-2.6.16-rc5/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-04 13:25:48.000000000 +0200
> > > +++ linux-2.6.16-rc5-k1/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-18 09:45:24.000000000 +0200
> > > @@ -1978,7 +1978,8 @@ static struct scsi_host_template sym2_te
> > > .eh_bus_reset_handler = sym53c8xx_eh_bus_reset_handler,
> > > .eh_host_reset_handler = sym53c8xx_eh_host_reset_handler,
> > > .this_id = 7,
> > > - .use_clustering = DISABLE_CLUSTERING,
> > > + .use_clustering = ENABLE_CLUSTERING,
> > > + .max_sectors = 0xFFFF,
> >
> > Strictly speaking, the clustering bit is unrelated.
>
> It is related. The number of scatter/gather segments in the SCSI HBAs is
> limited. In order to fit the large requests into these limits, the
> adjacent pages (that have been split to bios) have to be recombined.
As James also described, you cannot rely on clustering at all to get you
bigger requests. I'm thinking you are perhaps misunderstanding what that
option does - it only potentially limits the number of sg segments in a
request, if the pages happen to be physically contig. So it'll only help
you to a certain degree, IFF the pages are indeed adjacent in memory.
It's an extra optimization that isn't deterministic at all.
And it definitely is unrelated to how many sectors you can support. As
such, these two unrelated patches should submitted seperately. The
clustering bit is potentially more dangerous, as the hardware can have
all sorts of 'issues' related to the size and alignment of sg segments.
> > I seem to recall
> > Gerard years ago talking about some sym chips that did not like
> > clustering, hence it was disabled.
> >
> Facts?
I'm just reporting what Gerald (who definitely knows this hardware very
well) told me, when I submitted a patch to him enabling clustering a
long time ago. This might have been about 5 years ago, I seem to recall
it happening when I used a un ultra10 workstation which had a sym scsi
board.
--
Jens Axboe
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 20:43 ` Jens Axboe
@ 2006-03-01 22:24 ` Kai Makisara
2006-03-02 3:57 ` Matthew Wilcox
2006-03-02 7:29 ` Jens Axboe
0 siblings, 2 replies; 14+ messages in thread
From: Kai Makisara @ 2006-03-01 22:24 UTC (permalink / raw)
To: Jens Axboe; +Cc: matthew, linux-scsi
On Wed, 1 Mar 2006, Jens Axboe wrote:
> On Wed, Mar 01 2006, Kai Makisara wrote:
> > On Wed, 1 Mar 2006, Jens Axboe wrote:
> >
> > > On Tue, Feb 28 2006, Kai Makisara wrote:
> > > > This patch enables clustering and sets max_sectors to 0xffff to enable
> > > > reading and writing of large blocks with tapes (and large transfers with
> > > > sg). This change is needed after the sg and st drivers started using
> > > > chained bios through scsi_request_async() in 2.6.16.
> > > >
> > > > Signed-off-by: Kai Makisara <kai.makisara@kolumbus.fi>
> > > >
> > > > --- linux-2.6.16-rc5/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-04 13:25:48.000000000 +0200
> > > > +++ linux-2.6.16-rc5-k1/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-18 09:45:24.000000000 +0200
> > > > @@ -1978,7 +1978,8 @@ static struct scsi_host_template sym2_te
> > > > .eh_bus_reset_handler = sym53c8xx_eh_bus_reset_handler,
> > > > .eh_host_reset_handler = sym53c8xx_eh_host_reset_handler,
> > > > .this_id = 7,
> > > > - .use_clustering = DISABLE_CLUSTERING,
> > > > + .use_clustering = ENABLE_CLUSTERING,
> > > > + .max_sectors = 0xFFFF,
> > >
> > > Strictly speaking, the clustering bit is unrelated.
> >
> > It is related. The number of scatter/gather segments in the SCSI HBAs is
> > limited. In order to fit the large requests into these limits, the
> > adjacent pages (that have been split to bios) have to be recombined.
>
> As James also described, you cannot rely on clustering at all to get you
> bigger requests. I'm thinking you are perhaps misunderstanding what that
> option does - it only potentially limits the number of sg segments in a
> request, if the pages happen to be physically contig. So it'll only help
> you to a certain degree, IFF the pages are indeed adjacent in memory.
> It's an extra optimization that isn't deterministic at all.
>
The computer is deterministic. In this case we know what pages are
physically contiguous because they have been allocated in kernel space
with alloc_pages(). The st driver has made sure that the buffer can be
described with the scatter/gather list supported by the SCSI HBA.
In general, you should not rely on coalescing. Here I agree with you. If
you know what the buffer structure is, then you may be able to rely on
coalescing.
> And it definitely is unrelated to how many sectors you can support. As
> such, these two unrelated patches should submitted seperately. The
> clustering bit is potentially more dangerous, as the hardware can have
> all sorts of 'issues' related to the size and alignment of sg segments.
>
They are related if in the sense that both are needed in order to support
large requests. However, I don't mind if someone splits those changes into
two patches if both go in. Otherwise I don't recommend sym53c8xx_2 and/or
sym53c8xx chips for any serious work any more.
> > > I seem to recall
> > > Gerard years ago talking about some sym chips that did not like
> > > clustering, hence it was disabled.
> > >
> > Facts?
>
> I'm just reporting what Gerald (who definitely knows this hardware very
> well) told me, when I submitted a patch to him enabling clustering a
> long time ago. This might have been about 5 years ago, I seem to recall
> it happening when I used a un ultra10 workstation which had a sym scsi
> board.
>
A bit vague but I guess this is all we know. Unless Matthew knows more?
--
Kai
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 22:24 ` Kai Makisara
@ 2006-03-02 3:57 ` Matthew Wilcox
2006-03-02 7:29 ` Jens Axboe
1 sibling, 0 replies; 14+ messages in thread
From: Matthew Wilcox @ 2006-03-02 3:57 UTC (permalink / raw)
To: Kai Makisara; +Cc: Jens Axboe, linux-scsi
On Thu, Mar 02, 2006 at 12:24:27AM +0200, Kai Makisara wrote:
> > I'm just reporting what Gerald (who definitely knows this hardware very
> > well) told me, when I submitted a patch to him enabling clustering a
> > long time ago. This might have been about 5 years ago, I seem to recall
> > it happening when I used a un ultra10 workstation which had a sym scsi
> > board.
> >
> A bit vague but I guess this is all we know. Unless Matthew knows more?
Alas, I don't know the hardware very well. I try my hardest to fix
bugs and clean the driver up, but I'm no Gerard. I do have the chip
documentation, but it's not exactly conducive to answering these kinds
of questions. Also, the PDFs don't include the errata that Gerard
clearly had access to.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-01 22:24 ` Kai Makisara
2006-03-02 3:57 ` Matthew Wilcox
@ 2006-03-02 7:29 ` Jens Axboe
2006-03-04 8:16 ` Kai Makisara
1 sibling, 1 reply; 14+ messages in thread
From: Jens Axboe @ 2006-03-02 7:29 UTC (permalink / raw)
To: Kai Makisara; +Cc: matthew, linux-scsi
On Thu, Mar 02 2006, Kai Makisara wrote:
> On Wed, 1 Mar 2006, Jens Axboe wrote:
>
> > On Wed, Mar 01 2006, Kai Makisara wrote:
> > > On Wed, 1 Mar 2006, Jens Axboe wrote:
> > >
> > > > On Tue, Feb 28 2006, Kai Makisara wrote:
> > > > > This patch enables clustering and sets max_sectors to 0xffff to enable
> > > > > reading and writing of large blocks with tapes (and large transfers with
> > > > > sg). This change is needed after the sg and st drivers started using
> > > > > chained bios through scsi_request_async() in 2.6.16.
> > > > >
> > > > > Signed-off-by: Kai Makisara <kai.makisara@kolumbus.fi>
> > > > >
> > > > > --- linux-2.6.16-rc5/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-04 13:25:48.000000000 +0200
> > > > > +++ linux-2.6.16-rc5-k1/drivers/scsi/sym53c8xx_2/sym_glue.c 2006-02-18 09:45:24.000000000 +0200
> > > > > @@ -1978,7 +1978,8 @@ static struct scsi_host_template sym2_te
> > > > > .eh_bus_reset_handler = sym53c8xx_eh_bus_reset_handler,
> > > > > .eh_host_reset_handler = sym53c8xx_eh_host_reset_handler,
> > > > > .this_id = 7,
> > > > > - .use_clustering = DISABLE_CLUSTERING,
> > > > > + .use_clustering = ENABLE_CLUSTERING,
> > > > > + .max_sectors = 0xFFFF,
> > > >
> > > > Strictly speaking, the clustering bit is unrelated.
> > >
> > > It is related. The number of scatter/gather segments in the SCSI HBAs is
> > > limited. In order to fit the large requests into these limits, the
> > > adjacent pages (that have been split to bios) have to be recombined.
> >
> > As James also described, you cannot rely on clustering at all to get you
> > bigger requests. I'm thinking you are perhaps misunderstanding what that
> > option does - it only potentially limits the number of sg segments in a
> > request, if the pages happen to be physically contig. So it'll only help
> > you to a certain degree, IFF the pages are indeed adjacent in memory.
> > It's an extra optimization that isn't deterministic at all.
> >
> The computer is deterministic. In this case we know what pages are
> physically contiguous because they have been allocated in kernel space
> with alloc_pages(). The st driver has made sure that the buffer can be
> described with the scatter/gather list supported by the SCSI HBA.
>
> In general, you should not rely on coalescing. Here I agree with you.
> If you know what the buffer structure is, then you may be able to rely
> on coalescing.
Yes your case is special, since you are the producer of the pages and
thus can control them.
> > And it definitely is unrelated to how many sectors you can support. As
> > such, these two unrelated patches should submitted seperately. The
> > clustering bit is potentially more dangerous, as the hardware can have
> > all sorts of 'issues' related to the size and alignment of sg segments.
> >
> They are related if in the sense that both are needed in order to support
> large requests. However, I don't mind if someone splits those changes into
> two patches if both go in. Otherwise I don't recommend sym53c8xx_2 and/or
> sym53c8xx chips for any serious work any more.
For debug and bisect reasons alone I think it should be two patches. The
cluster change cannot go into 2.6.16, it's way too late for that (I
would not personally be against the max_sectors changes...).
> > > > I seem to recall
> > > > Gerard years ago talking about some sym chips that did not like
> > > > clustering, hence it was disabled.
> > > >
> > > Facts?
> >
> > I'm just reporting what Gerald (who definitely knows this hardware very
> > well) told me, when I submitted a patch to him enabling clustering a
> > long time ago. This might have been about 5 years ago, I seem to recall
> > it happening when I used a un ultra10 workstation which had a sym scsi
> > board.
> >
> A bit vague but I guess this is all we know. Unless Matthew knows more?
So I dig out the old mail from Gerard from April 2000, it's pasted
below.
--------------
Hi Jens,
On Sun, 23 Apr 2000, Jens Axboe wrote:
> Hi Gerard,
>
> I've been using my onboard adaptec 7890, but since it does not want to
> do TCQ for me, I switched adapters and I'm now using your driver. Now
> I'm wondering, what is the reason that you disable clustering in the
> host template??
This had been numerous historical reasons for that. :)
(Note that the 53c7,8xx driver also disables clustering and clustering
does guarantee larger IO to be always possible given a max number of SG
entries).
The initial reasons were the following:
1) The ncr drivers uses a simple array of MOVE scripts instructions and
clustering the data to decrease the number of SG entries didn't
significantly increase performances.
2) Instead, th ncr/ncr53c8xx driver tried to use 512 byte sized SG
entries if possible with a JUMP between each MOVE in order to avoid
phase mismatch interrupts.
The 53c7,8xx uses the same trick, but just leave the SG sizes unchanged
(was 1K with 1K block FS).
I have removed this complexity from the sym53c8xx driver (it seemed to
me rather a loss than a win), for numerous reasons, notably:
1) The handling of unaligned transfer (notbaly WSR bit) for Wide transfer
would have still complexified the SCRIPTS.
2) Starting with the 896, phase mismatches are handled from SCRIPTS and
the additionnal JUMP between each MOVE was a penalty.
3) The trick was only interesting for small IOs but clever disk firmwares
mostly disconnect just after the COMMAND phase for small IOs, making
the MOVE+JUMP trick just useless complexity.
On paper, there is no reason to not cluster SG entries with the sym53c8xx
driver, but since it should not increase significantly performances nor
guarantee larger IO to be always possible, my opinion is that there is no
strong reason to enabled clustering by default and for Ultra <= 2. (For 4K
FS / 80 MB/s, I am sure there is no significant performance impact if
clustering is not enabled.
The other reason that let me hesitate to enable clustering by default, for
the moment, is that the size of a scatter entry must be limited to 16 MB.
The size is not a problem, but I would prefer to _also_ guarantee that a
scatter entry will not cross a 24 bit boundary given the chip core
performing 24 bit calculation. By experience, I avoid everything that may
trigger chip bugs and such a crossing smelt so to me. Btw, the 896 rev. 1
has such a bug documented.
Btw, in my opinion, a _usable_ clustering scheme must allow to specify:
1) A maximum length for scatter entries.
2) A bus physical boundary address to not cross within all
scatter entries.
AFAIR, the Linux approach for clustering does not allow that.
However, as you can see in the sym53c8xx source, there is some provision
that catches the 16MB boundary problem of the 896 rev. 1, but this hasn't
been ever triggerred. If, on the other hand, your controller isn' a 896
rev. 1, clustering should normally not make problems (based on currently
documented chip erratas).
Speaking about current `sym' drivers for both Linux and FreeBSD. The use
of the kernel Bus Dma abstraction in both O/Ses does not prevent the Dma
interface from performing SG clustering. :-)
The clustering of SGs is, in fact, actually in use with the FreeBSD
-current sym_hipd driver (which is similar to sym53c8xx in design) since a
couple of weeks, with some provision for the 896 rev. 1 errata. Such a
provision for 896 rev.1 also exists under Linux, obviously. No problem has
been reported, as expected. :-)
As you can see, the use of clustering is currently under experimentation,
somewhere. In fact, for Ultra-160, the clustering of SGs may have
mesurable good effects on performances, IMO, and I actually want it to be
used.
Regards,
Gérard.
----------------------
--
Jens Axboe
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] SCSI sym53c8xx_2: bigger transfer limits
2006-03-02 7:29 ` Jens Axboe
@ 2006-03-04 8:16 ` Kai Makisara
0 siblings, 0 replies; 14+ messages in thread
From: Kai Makisara @ 2006-03-04 8:16 UTC (permalink / raw)
To: Jens Axboe; +Cc: matthew, linux-scsi
On Thu, 2 Mar 2006, Jens Axboe wrote:
> On Thu, Mar 02 2006, Kai Makisara wrote:
> > On Wed, 1 Mar 2006, Jens Axboe wrote:
> >
> > > On Wed, Mar 01 2006, Kai Makisara wrote:
...
> > > And it definitely is unrelated to how many sectors you can support. As
> > > such, these two unrelated patches should submitted seperately. The
> > > clustering bit is potentially more dangerous, as the hardware can have
> > > all sorts of 'issues' related to the size and alignment of sg segments.
> > >
> > They are related if in the sense that both are needed in order to support
> > large requests. However, I don't mind if someone splits those changes into
> > two patches if both go in. Otherwise I don't recommend sym53c8xx_2 and/or
> > sym53c8xx chips for any serious work any more.
>
> For debug and bisect reasons alone I think it should be two patches. The
> cluster change cannot go into 2.6.16, it's way too late for that (I
> would not personally be against the max_sectors changes...).
>
The max_sectors change alone does not change anything for this driver.
The maximum number of scatter/gather segments is set to 96 and this limits
the transfers to 384 kB (with 4 kB pages) which is below the 1020 kB limit
resulting from increasing max_sectors.
For tapes, I don't think it is worth investigating if the number of
scatter/gather segments can be increased unless we get a specific user
request. AFAIK, block sizes above 256 kB are rare and if some application
needs larger size, it may well need beyond 1 MB.
> > > > > I seem to recall
> > > > > Gerard years ago talking about some sym chips that did not like
> > > > > clustering, hence it was disabled.
> > > > >
> > > > Facts?
> > >
> > > I'm just reporting what Gerald (who definitely knows this hardware very
> > > well) told me, when I submitted a patch to him enabling clustering a
> > > long time ago. This might have been about 5 years ago, I seem to recall
> > > it happening when I used a un ultra10 workstation which had a sym scsi
> > > board.
> > >
> > A bit vague but I guess this is all we know. Unless Matthew knows more?
>
> So I dig out the old mail from Gerard from April 2000, it's pasted
> below.
>
[the long message cut from this reply]
Thanks for digging up this message. I read it so that most sym53c8xx chips
have no problems with arbitrary page alignments and counts if the segment
length is below 16 MB. However, there are legitimate questions suggesting
that this may not apply to all chips and versions. It is best not to
change the limits in the released kernel until the questions have been
addressed (which will probably never happen considering the age of the
chips series, number of different versions, etc.; sad but we have to
accept this). Those who really need the changes (like me) can then edit
the driver and take the risk.
--
Kai
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2006-03-04 8:13 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-02-28 19:14 [PATCH] SCSI sym53c8xx_2: bigger transfer limits Kai Makisara
2006-02-28 20:41 ` Matthew Wilcox
2006-03-01 15:29 ` Jens Axboe
2006-03-01 15:39 ` James Bottomley
2006-03-01 15:54 ` Jens Axboe
2006-03-01 16:00 ` Douglas Gilbert
2006-03-01 16:11 ` Jens Axboe
2006-03-01 16:17 ` James Bottomley
2006-03-01 20:34 ` Kai Makisara
2006-03-01 20:43 ` Jens Axboe
2006-03-01 22:24 ` Kai Makisara
2006-03-02 3:57 ` Matthew Wilcox
2006-03-02 7:29 ` Jens Axboe
2006-03-04 8:16 ` Kai Makisara
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).